I don't explain anything that doesn't require explanation, which means there's alot of "wasted" lore. The players came to this world in the middle of a cold period, and for all they knew it was inhabited by a race of hairy manly-men and their androgynous wives. If they had tried to fuck one of the herms I suppose I would have had some explaining to do, but at that point they would hardly be able to accuse me of having a fetish. They did encounter a herm/herm couple while there. They didn't seem interested to know how they had children. The couple was poor (they can't venture out of the house and work for as long as the males can) and their infant mortality rate was high (their female offspring would die soon after birth from the ambient chill) and my players just decided both were due to oppressive patriarchal structures in place which are keeping them down.
I intend to have them return to the planet in 200 years, in the middle of a warm period. The situation will be the same (dominated by females, herms second-class citizens) but my expectation is that my players will feel it's much more fair because now the ones on the bottom are the "males".
I kind of hate my players for a number of reasons I won't get into here.