A number of studies have found differences in the brain that may be related to sexual orientation.
Because of this, studies of gender identity and the brain need to carefully control for sexual orientation.
It is important to remember that the majority of trans men (born female) are primarily attracted to women while only about 5% of cis women are primarily attracted to women. Thus if you find differences between trans men and a randomly selected group of females, the differences could be due to gender identity or sexual orientation.
Similarly, about half of trans women (born male) are attracted to men while only about 5% of cis men are primarily attracted to men. Again, differences found in studies of the brain could be due to gender identity or sexual orientation.
So far, I have found no studies of the brain and gender identity that included gay and lesbian controls.
Many studies discuss the sexual orientation of the trans people; some studies look only at trans women who are attracted to men or trans women who are attracted to women, but that is only half of what needs to be done.
A group of trans women who are all attracted to men is still different from your average group of cis men in two ways – gender identity and sexual orientation.
The studies listed below are proof this issue matters. We know that sexual orientation can affect the brain.
I have not read these studies, this is just a list of links for you to enjoy.
A fiber tract that is larger in women than in men was even larger in gay men. (But the gay men had died of AIDs).
The interstitial nuclei of the human anterior hypothalamus: an investigation of variation with sex, sexual orientation, and HIV status by W. Byne, S. Tobet, L.A. Mattiace, M.S. Lasco, E. Kemether, M.A. Edgar, S. Morgello, M.S. Buchsbaum, L.B. Jones in Horm. Behav., 40 (2001), pp. 86–92.
HIV status affected the size of the INAH1 cell group. The INAH3 cell group was bigger in presumed straight males than females and contained more neurons. There was a trend for INAH3 to have a larger volume in straight males than gay males, but they had the same number of neurons in the INAH3.
The INAH3 cell group in the hypothalmus was bigger in men than women. It was also bigger in the brains of presumed straight men than in the brains of gay men.
Homosexual women have less grey matter in perirhinal cortex than heterosexual women by J. Ponseti, H.R. Siebner, S. Kloppel, S. Wolff, O. Granert, O. Jansen, H.M. Mehdorn, H.A. Bosinski in PLoS. ONE, 2 (2007), p. e762.
Women had more gray matter than men in a number of areas. In three areas straight women had more gray matter than lesbians; in one of these areas men also had less gray matter than women.
“The perirhinal cortex is located close to entorhinal cortex, hippocampus, parahippocampal gyrus and amygdala, and is known to be involved in a variety of functions like olfactory processing, memory encoding and spatial processing. These functions are related to the processing of sexual stimuli as well.”
PET and MRI show differences in cerebral asymmetry and functional connectivity between homo- and heterosexual subjects by I. Savic, P. Lindstrom in Proc. Natl. Acad Sci. U. S. A., 105 (2008), pp. 9403–9408.
Straight men and lesbians had a rightward cerebral asymmetry, gay men and straight women did not.
“Homosexual subjects also showed sex-atypical amygdala connections. In HoM, as in HeW, the connections were more widespread from the left amygdala; in HoW and HeM, on the other hand, from the right amygdala. Furthermore, in HoM and HeW the connections were primarily displayed with the contralateral amygdala and the anterior cingulate, in HeM and HoW with the caudate, putamen, and the prefrontal cortex. The present study shows sex-atypical cerebral asymmetry and functional connections in homosexual subjects.”
A nucleus in the hypothalmus is larger in gay men than straight men. A different nucleus that is sexually dimorphic was the same size in gay and straight men.
Corpus callosum anatomy in right-handed homosexual and heterosexual men by S.F. Witelson, D.L. Kigar, A. Scamvougeras, D.M. Kideckel, B. Buck, P.L. Stanchev, M. Bronskill, S. Black in Arch. Sex Behav., 37 (2008), pp. 857–863.
The isthmal area in the corpus callosum was larger for a group of gay men compared to straight men.
The citations for these studies are from the study Regional gray matter variation in male-to-female transsexualism by Luders E, Sánchez FJ, Gaser C, Toga AW, Narr KL, Hamilton LS, Vilain E. in Neuroimage. 2009 Jul 15;46(4):904-7. I reviewed the study here.