Freakonomics, Nudge, etc.: The “professional wrestling” of science

Megan Higgs and I recently published an article, Interrogating the “cargo cult science” metaphor, arguing against the use of the term “cargo cult science” to describe ritualistic empty research. (Regular readers of the blog will recall many examples of this sort of bad science, including beauty and sex ratio, ovulation and voting, himmicanes, air rage, ages ending in 9, subliminal smiley faces, ummmm . . . here’s a partial list.)

One of our issues with the “cargo cult” slogan was that the original cargo cult members were detached from Western society, whereas many of the modern perpetrators of junk science are well plugged in to the science and media establishment. For example, the authors of Nudge teach at leading universities, one of them has held government posts, another has founded an asset management firm, and they are regularly featured uncritically in the national news media. The author of the notorious beauty-and-sex-ratio paper teaches at a leading university and his work has been featured on Freakonomics. Several of these horrible papers have been published by the National Academy of Sciences, which I guess is the leading organization of scientists in the country. So, yeah, terrible work all around but not being conducted by remote participants in a “cargo cult.” As I wrote a few years ago, the calls are coming from inside the house.

So what’s a better term, if we’re not going to call it “cargo cult science”? One idea is “ritualistic science,” which indeed captures part of it, the idea that research methods such as hypothesis testing and causal identification, which have real scientific motivation and can still be employed in useful ways, have become totems which are being used not as tools within scientific investigation but more as replacements for scientific thinking. Instead of careful measurement, strong theory, and openness to being surprised, we see thoughtless measurement, empty speculation, and the use of methods of hypothesis testing and causal identification to reify existing beliefs. So, yeah, ritual is part of it. But I don’t think the rituals are the key part.

Another term that could work would be “parasitic science” or “mimicry science.” The idea here is that junk science lives off the reputation of existing and past good science. One place we see this is the fake studies cited by the U.S. Department of Health and Human Services to support their political agenda. These are papers published in fake journals, or in some cases papers that never existed, hiding in plain sight amidst the millions of real research papers (of varying quality) in the scientific literature. “Parasitic” is a tricky term, though, in that, in a sense, all science is parasitic (“standing on the shoulders of giants,” etc.). The point here is that junk science doesn’t just feel like real science to its perpetrators (that’s the “cargo cult” and “ritualistic” part); it also is designed, sometimes cannily so, to look like real science to outsiders.

A completely different analogy then came to mind: professional wrestling.

This captures a couple of key features that distinguish modern junk science. The first is its public-facing nature: junk science is very much a performance of science. Here we talk more about the social science than the natural sciences, so the props are p-values, randomized experiments, regression discontinuity analyses, and robustness checks rather than test tubes, lab coats, and safety goggles–but either way these paraphernalia are there for public consumption. Another similarity is the close connection with the media–for junk science it’s the news media and for wrestling it’s the entertainment media–, but in either the case the rules of the game are modified to fit the promotional media.

A third similarity is that, to do junk science well, it helps to know something about real science, in the same way that professional wrestling may be “fake,” but the people doing it still are wrestling, they’re strong and coordinated and have to practice difficult moves. Take Brian Wansink and Dan Ariely, for example, two media darlings who, through no fault of their own, seem to have ended up being authors on fraudulent scientific papers. These two guys might not know a lot about conducting and analyzing experiments to gain reproducible scientific knowledge, but they’re true experts in writing papers that are convincing enough to get published, and they’re true experts in publicizing their work. The point is, they have some actual science-adjacent skills–indeed, enough skills to snag plum academic jobs.

OK, pro wrestling is not such a great fit either. For one thing, wrestling is scripted with heroes and villains; the science script of PNAS, NPR, Gladwell, Ted, Carroll, Nudge, etc., is pretty much all heroes, with the only villains being picky outsiders like me, the “second stringers . . . Stasi . . . terrorists” who dare to question our betters. But science critics do not play the role in pop science that “heels” do in wrestling. We’re not supposed to be part of the story line at all, and with rare exceptions the mainstream will ignore us rather than address our concerns.

Also, junk science is, as noted above, parasitic on real science. I would not say that there is something called “real wrestling” that professional wrestling is a fake version of.

So I’m still searching for a good analogy for junk science that captures all the features discussed above.