Recently professors James Lindsay, Helen Pluckrose and Peter Boghossian sent out 20 fake papers to various peer-reviewed journals. Six of them were published. This, say various news outlets, is a real black eye for gender scholarship and, perhaps, for academia in general. On closer examination, ThisWeekInStupid concludes that the real hoax was the one in which serious journalists were convinced that influential journals were publishing shoddy, but honest, scholarship.
TWIS did a thorough review of the paper, reviews and revisions of the research titled “An Ethnography of Breastaurant Masculinity: Themes of Objectification, Sexual Conquest, Male Control, and Masculine Toughness in a Sexually Objectifying Restaurant” published in Sex Roles on September 19, 2018. The paper was first submitted to and rejected by Men and Masculinities. This was the only paper TWIS examined. It’s reasonable to expect that some papers were more and some were less legitimate than this paper. TWIS did not explicitly select this paper to support a pre-determined conclusion.
All of the paper and reviews can be found here. Let me first say that it was brave and honest and sincere of the perpetrators of the hoax to publish the papers and reviews. Bravo. But they didn’t pull off what they think they did.
The Con
The original submission, to judge by the reviewer comments, was a few pages of jargon-y reflections on the fake author’s Thursday nights with the bros at the local Hooters. It was peppered with broad generalizations and hair-on-fire hyperbole of the kind that RWNJs attribute to SJWs. It also featured liberal (see what I did there?) use of the brand-new and super-clever label “breastaurant.” The fake authors were very proud of this invention and the various journalists were quite taken in by it as well. But here’s what actually did and didn’t happen.
What didn’t happen
Contrary to coverage by non-academic papers, the journals were not fooled by the original haphazard summary of conversations among 7 men at a Hooter’s restaurant after jiu jitzu class. Both journals, and all but one reviewer, heavily criticized the methods section. They complained that a single restaurant, a single coder and a participant-observer made the findings dubious. Their responses included dozens of questions about how the quotes used were selected, where and when observations and interviews occurred and how the observations were connected to the conclusions. They questioned the time frame, the underlying assumptions about the state of masculinity in the broad culture, the selection of quotes, the coding and the mind-reading of the participants.
Both journals were quick to put the kaibosh on the sweeping conclusions in the original paper. The final paper was reduced (appropriately) to a participant ethnography of a single site with appropriate caveats and qualifications. The authors of the fake paper would do well to explore the theories of ethnographic research as reviewers repeatedly reminded them. Their expectation that a single site ethnography would be enough to codify broad social conclusions indicates they have a lot to learn. The original theoretical framework, grounded theory, was rejected by both the reviewers and the journal.
The reviewers were not fooled by the dense jargon in the paper with one reviewer “encourag[ing] the author to consider word choice and ask himself when he might be inappropriately applying jargon that actually obscures the specific practices at play.” Another was even less convinced.
The use of “pastiche hegemony” and “ersatz sexual availability” is unnecessary and relieves the author of having to be more specific. They sound smart but really stand in for more tangible identification and explanations of practices. Drop these concepts, or refer to them and then move on in favor of a clearer analysis of exact observed practices.
Ouch! In the original response, the editor of Sex Roles summarized
The rigor of your research rests in sound methods, not in obscure language.
In other words, cut out the abstruse posturing. You just sound like morons.
Throughout, reviewers were skeptical of the conclusions drawn by the authors. Here are some samples from reviewers:
Whilst and interesting an engaging topic, it is unclear at the outset what novel insight or contribution this paper will be making. Much work will be required before it is of publishable standard.
The author(s) assert that “this study is mostly theoretical in its approach, but it is bolstered by comparison against my own empirical data” (p. 14, ll. 57-59). This is an unnecessary assertion. This paper may be defined as empirical by conventional standards.
There are a lot of assumptions in this paper, some of which the author uses to build a case for his work and analysis. Assumptions are a good indicator that data is thin, and this is certainly the case in this paper.
The reviewers saw the data as inadequate to draw the conclusions the authors were drawing. A single site participant-observer ethnography is just that and doesn’t give license to draw sweeping conclusions.
Finally, looking at the 20-paper hoax as a whole, the authors did not manage to achieve an impressive publication record for 3 researchers working in concert and skipping all the pesky data collection steps. The authors were quoted saying that 6 articles is enough to earn a professor tenure. That distant laughter your hear is all of the professors in the known universe. Six co-authored papers in the Journal of Poetry Therapy gets exactly no one tenured. And one article based on falsified data is enough to get you canned.
It’s important to note here the background of the fake authors. Peter Boghossian is a professor of philosophy. James Lindsey is a mathematician. These disciplines are unique in that conclusions usually are not, and need not be, backed by empirical data. As such, the authors likely don’t have much understanding of the importance of reliable data or much appreciation for the effort involved in gathering data. But biologists know. And physicists. And astronomers and chemists and geologists and political scientists and engineers and even advertising executives know that there’s a lot of trust in research. Inventing data doesn’t make you clever. It makes you a hack and a liar. A sociologist with whom I discussed this news said, “Big deal. They skipped the hard part. If I could fabricate all my data, I’d win the Nobel Prize!”
What did happen
The final response from Sex Roles indicates that this is the first ever ethnography published in their usually quantitative journal. She also mentioned that a submission by a historian is novel. One can imagine an editor, very excited to expand the (very old) journal into a new area, green-lit a project slightly outside her expertise. The editors and reviewers for Men & Masculinities, where a great deal more ethnography is published, were not fooled. Sex Roles assigned a junior editor more familiar with ethnography to bring the paper up to standard. The final paper is not good and one can feel the authors intentionally pushing the envelope trying to sneak in the maximum allowed number of outlandish claims. To our eyes, this is a case of a junior editor getting snowed by a senior professor and the enthusiasm of her superior.
The experiment actually reveals some very wonderful things about academia. In the first place, Richard Baldwin, an emeritus history professor from Gulf Coast State College, gets legitimate consideration from a sociology and psychology journal neither of which had ever published his work. Gender scholarship is not an isolated, exclusive club. The interdisciplinary interaction among historians, economists, psychologists, anthropologists and scholars of gender, race and class is a boon to scholarship. But, contrary to the conclusions of Dr. William Eggington the willingness of Sex Roles to entertain scholarship using a variety of methods from a variety of fields is an important check on academic navel-gazing and not an indication of over-specialization.
In the second place, the journals were quite careful about participant consent and potential taint in the research due to personal relationships. The original paper didn’t mention whether participants were aware they were being studied or the relationship of the various participants to the researcher. At the insistence of reviewers, the paper was revised to (dishonestly) indicate that participants, including wait staff, gave consent, that interviews were recorded, etc.
Finally, the ignorance of the authors of concepts and practices in gender studies was ruthlessly laid bare by reviewers. Throughout the reviews, the authors were taken to task for imprecise use or misuse of terms–“coding,” “discourse,” “theme,” “scripts,” “neoliberal-patriarcho-capitalist” (for heaven’s sake), etc.
There is one quite laughable assertion by the fake authors in response to their intellectual paddling at the hands of Men & Masculinities.
Nota bene: The second reviewer is particularly harsh, it is reasonable to conclude that the paper was rejected on the review of the second reviewer. Upon our reading of this review, it seems that the second reviewer wanted us to write a completely different paper, focused more on themes of feminism than on those of masculinity, even though having done so would likely have taken the paper outside of the scope of the journal Men & Masculinities for which it was being reviewed.
This is a bizarre reading of the reviewer’s comments which were actually a criticism of the paper’s methods and the objectivity of the (imaginary) researcher. The review included this passage:
The author really ignores the women as research participants. There needs to be a better, empirical and theoretical explanation for why the author focuses solely on men when the women are clearly key to the interactional processes by which (a particular kind of masculinity – be clear) is supported through women’s work. Ignoring women as participants risks further objectifying them.
Preach! (Did you notice how he authors got a bonus lesson? There is more than one kind of masculinity! There might even be enough to say about masculinity to fill a whole journal!) The reviewer also made this comment which argues that the paper found the women in the space occupying the only role that the research design allowed them to:
Ironically, the author brings women into his analysis only in terms of how they act as “identity resources” for men—that is, he seems to argue one of his main findings is that women are identity resources, but also reduces them to such by not conducting an analysis of women’s agency. Toward the end of the paper, he even suggests that the “power always flows from customer to server”—i.e., men have and exert power over women. This analysis is reductionist.
The ignorance of the hucksters is fully on display here. Naturally, thought the rubes, a journal called “Men & Masculinities” is a place where the behaviors and assumptions of men can be safely analyzed without the pesky intrusion of the sentient feminine. Isn’t there any space left where men can conduct research about manly men in men’s spaces without some feminist interjecting to insist that the women in the story are people!?
Let’s also take a minute to appreciate that the pandering authors were trying to toe the party line with their comment that men always have the power. Meanwhile, the journal, ostensibly packed to the gills with single-minded feminists and “grievance scholars,” stops them short, pointing out that, even in the relatively uncomplicated space of Hooters, the power dynamics are more complicated.
The fake authors submitted shoddy research. On being found out, they doubled down on deception, adding fake details to their methods to address reviewers’ accurate criticisms. In the end, they effectively falsified data, something which would be no more easily detected in physics or engineering and, which, in fact, happens all the time. What’s more, as they began by trying to broadly extrapolate from woefully inadequate data they were thwarted and called to task. That was the prank, you see. “I can get these idiots to publish my ramblings on hanging out at Hooters with my bros. Tee-hee.” But after review, the corrected manuscript was pared down to present the findings as honestly and circumspectly as the deception of the authors allowed. Your participant-observations of your bros are legitimate if you follow established research protocols and forthrightly declare your methods. This is how participant ethnography happens.
TWIS did not take time to review all 20 articles, or even the 7 which were published. We refuse to buy into the Gish Gallop that assumes, without examination, that all represent poor scholarship or to be buried by the proof by verbosity. But, if we assume all of the papers followed a path similar to that of the “Hooters” paper, academia passes the test put to it.
