This study is seriously flawed and overstates its conclusion that surgery is associated with improved mental-health related quality of life for trans women.
The authors of the study surveyed 247 trans women. They administered a questionnaire made from three surveys. The first part asked about the person’s age and their transition, the second part asked six questions related to their face, and the third part was the San Francisco 36-question health questionnaire (SF36v2).
The authors found that:
1. Trans women who had had any form of surgery had mental health outcomes equal to the general female population; trans women who had not had any surgery had mental health outcomes that were worse than the general population. It did not make any difference what type of surgery they had; FFS, gender reassignment surgery, or both.
2. All the trans women in their study had better physical health than the general female population.
3. Trans women who had had FFS had significantly higher scores on the six question Facial Feminization Survey than trans women who had not had FFS.
The biggest hole in this study (and you could drive a truck through it) is that the trans women who had had surgery were significantly different from the trans women who had not had surgery. The differences were important and relevant to the study’s results.
a) 95% of the surgery group were also taking hormones while only 66% of the non-surgery group were taking hormones.* It may be that the hormones and not the surgeries were responsible for the improved well-being.
b) 54% of the surgery group had transitioned more than five years ago while only 24% of the non-surgery group had transitioned more than five years ago.* It may be that being further along in transition was responsible for the improved well-being. There are many reasons this could be true including more time for family adjustment, having had more therapy, or knowing more about how to present as a woman.
It could also be that some people find that transition does not help them in the first few years of transition and drop out; this might mean that the group of people who are further along in transition does not include as many people who have difficulties with transition.
c) Facial feminization surgery and gender reassignment surgery are expensive and are usually paid for by individuals in the United States. It may be that the surgery group was significantly richer than the non-surgery group.
The finding that trans women had better physical health than the general female population is a baffling one. It suggests that the group of trans women who answered the survey were significantly different from the general population.
The finding also raises the question of whether you would expect the trans women who answered this survey to have a better mental health related quality of life than the general public.
Which brings us to the problems with the sample. The size of the sample in this study is a good one: 247 people, 100 who had surgery, 147 who did not. The sample may not be representative, however.
Participants were recruited from people who had received care from an FFS surgeon or clinic and people who were involved in transgender support groups or organizations. Most of the surveys were given online, but a paper version was given at a transgender health conference. People who have seen a surgeon are more likely to be well-off; FFS costs mentioned on one forum ranged from $23,000 to $64,000. The low end of the spectrum was for surgery abroad.
If the trans women answering the survey were in fact richer than the general public, it would explain why they had better physical health than most women. It might also mean that they should have had a better mental health quality of life than most women.
The authors do not discuss the order of the questions in the survey, but there may be another potential problem in the survey, depending on how it was done. Asking people questions about themselves can influence how they do on a test. It might be that asking people if they have had surgery or how long they have been on hormones affects their mood. Patients who want to have surgery but have not yet had it might feel sad or less confident. This might affect answers to questions about things like how happy they were in the last week. It matters if the demographic questions were at the end or the beginning.
Similarly, the questions about the face and its femininity might affect the mood of someone who wants FFS but cannot afford it.**
What about the results for the Facial Feminization Survey?
The authors say that one limitation of the survey is that it has not been tested in previous investigations. It was adapted from a survey originally designed for pre- and post-operative evaluations of the same patient. It is also a fairly brief questionnaire with only six items.
On the other hand, the questions are highly relevant ones that look at feelings about your face and how those feelings affect your life. It looks like a reasonable measurement tool.
Unfortunately, the data for this part of the survey has the same problem as the quality of life section. There are important and fairly large differences between the FFS group and the non-FFS group.
The study combines the groups that had FFS or FFS and gender reassignment surgery into one group and then combines the gender reassignment surgery only group with the non-surgery group. Because the non-surgery group is larger and includes so many more people who are not on hormones and/or are early in their transition, the non-FFS group ends up with more people who are not on hormones and/or are earlier in their transition.
So 93% of the people who had FFS were also on hormones while only 71% of the people who had not had FFS were taking hormones.* This is still a large difference and raises the question of whether trans women on hormones are happier with their faces – or just happier in general. (For any-surgery versus no-surgery the numbers are 95% vs. 66%.)
In addition, 47% of the people who had FFS had transitioned more than five years ago compared to only 32% of the people who had not had FFS.* (For any-surgery versus no-surgery the numbers are 54% vs. 24%.)
Finally, 100% of the people who had FFS had had surgery while only 17% of the non-FFS group had had surgery. It could be that any type of surgery improved mental health and therefore feelings about the face. More likely, being able to afford any type of surgery could be associated with having more money. The FFS group might be richer than the non-FFS group.
So we can’t be sure whether the FFS group felt better about their face due to FFS, hormones, or length of time since they started their transition. There might also be some other important difference that made them happier with their faces such as having a larger income.
Another concern about the study is that this seems to have been a middle-aged group of people. The mean age for the no surgery group was 46. For the FFS only group, the mean age was 51, for the gender reassignment surgery only group the mean age was 50, and for the group who had had both types of surgeries, the mean age was 49. This sample does not seem to be representative of all trans women, although it is unclear how this would affect the results.
As in many studies of FFS, one of the authors of the study is an expert in facial feminization surgery. This may mean he has a bias in favor of believing that it works.
What can we conclude from this study?
We need more research in this area!
Despite everything, this research is an important first step. We have no other studies that attempt to quantify the effects of facial feminization surgery. As doctors have said in many other studies, we need this data,
This study could have been greatly improved if the authors had controlled for hormonal status and length of time since the beginning of transition. Perhaps they can still use the data to do this.
It would also have helped if the authors had asked questions about income, education, and occupation and controlled for these important variables.
As it is, we can not conclude anything about the benefits of facial feminization surgery from this study.
The study provides some support for the idea that something about transition is beneficial to trans women. Even here, though, there is still a possibility that another factor like income was the important one.
We need someone, ideally someone with a background in psychological research, to re-do this study or one like it using a better sampling technique and controlling for important factors.
*These numbers and other percentages were calculated by George Davis based on data tables provided in the study.
** The Facial Feminization Survey was adapted from an FFS outcomes evaluation form. The original form includes questions specifically related to whether or not the patient wants FFS or other surgery. I am assuming these questions were not used in the survey for this study; however, if they were, they might significantly affect the mood of people who want surgery but have not had it. This would be in addition to any possible effect from asking people how they felt about their face.