Gaydar Goes AI and Populism Comes to Science
A new study has stirred a public outcry.
Posted Sep 13, 2017
An upcoming study entitled “Deep Neural Networks Are More Accurate than Humans at Detecting Sexual Orientation from Facial Images,” authored by Yilun Wang and Michal Kosinski, has raised a great deal of controversy (see “AI That Can Determine A Person’s Sexuality From Photos Shows the Dark Side of the Data Age” for a very biased discussion). That is an impressive feat for an article that is not yet published. It has been accepted for publication by the Journal of Personality and Social Psychology (JPSP), the top journal in the field of social psychology and a journal of the American Psychological Association (APA).
In their five studies, they used facial images from public profiles that had been posted by users to dating websites in the United States. The main findings of their series of studies was that the computer program could correctly classify between gay and heterosexual men at a higher rate of accuracy than humans could, and that key indicators were facial morphology, expression, and grooming styles.
They eased the minds of their readers in their General Discussion section that contained a variety of disclaimers and cautions (“Importantly, we would like to warn our readers against misinterpreting or overinterpreting this study’s findings”). They spent three paragraphs (about two double-spaced pages) of the General Discussion section covering what they found and why it was important, then a great deal of effort apologizing for what the study didn’t do (about five double-spaced pages of disclaimers and limitations). For more evidence that this study went beyond the typical peer review, their acknowledgments thanked 13 individuals (including a former President of APA) for “their critical reading of the earlier version of this manuscript” in addition to thanking the actual peer reviewers of the manuscript at the journal. Such proportions are usually the hallmark of a solid scientific study that has survived rigorous peer review in a politically charged or controversial topic.
The findings are not even surprising, given the findings of Dr. Kerri Johnson at UCLA (e.g., Johnson, Gill, Reichman, & Tassinary, 2007; note that it was also published in JPSP) that show people can detect sexual orientation from gait. But clearly the scientific merit of this series of studies is not the issue.
My article here is not about Wang and Kosinski’s studies. It is about populism science. After public outcry from a variety of organizations, the American Psychological Association asked to review the Institutional Review Board application and approval from Stanford University’s IRB, citing potential ethical violations in the use of the images as stimuli. Coleen Flaherty has an excellent article on this incident (with a sensationalized title) in “AI Gaydar Study Gets Another Look.”
Having served as author of peer reviewed journal articles, reviewer of peer reviewed journal articles, Editor of a peer reviewed journal, and Chair of a Federalwide Assured Institutional Review Board, the steps taken by APA and JPSP here are not commonly taken. In this case, the study was approved by the Stanford University Institutional Review Board, which has a Federalwide Assurance from the U.S. Department of Health and Human Services (FWA00000935). That means the Federal Government has approved the Stanford IRB to enforce federal regulations on the protection of human subjects. For APA to take the extraordinary measures of reviewing the IRB application after IRB approval and scientific peer review is inappropriate. Publication of the paper is currently on hold while APA sorts this out to their satisfaction.
Wang and Kosinski’s study was the victim of political pandering. Science is better than that, and scientists certainly should be. The scientific peer review process must play out in its entirety, and unpopular findings should not be swept under the rug, but should be addressed. The science was completed, and we can’t unsee the results of these five studies. Hiding behind puffed up charges of research ethics violations doesn’t make a competing narrative a reality. The Editor of JPSP and the peer reviewers found the study to have scientific merit sufficient for inclusion in the top journal in social psychology. These are the facts of this case.
Johnson, K. L., Gill, S., Reichman, V., & Tassinary, L. G. (2007). Swagger, sway, and sexuality: Judging sexual orientation from body motion and morphology. Journal of Personality and Social Psychology, 93, 321-334.
Wang, Y., & Kosinski, M. (in press). Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. Journal of Personality and Social Psychology.