A self-portrait by Mexican artist Frida Kahlo has become the most valuable work by a female artist sold at auction.