How were your conceptions of gender and sex formed if not by social interaction with others?
One could ask the question another way - How can we form a personal feeling of gender outside of social structure?
One idea is that there is an element of gender that is somehow inherent. A lot of people don't like that, because to them it says that women's and men's brains are "different" and therefore one must be superior - or at least people can start arguing about who is superior. And I get it, it's dangerous - but we already know men's and women's brains are different, and that the brains of trans people are more like the gender they feel they are than their assigned gender.
Makes a crazy kind of sense, and that's what I'm going with since it's backed up by the sources I showed you and that's about as good as I can get without actually studying to be a brain researcher.