Cultural Caviar

The Replication Crisis and the Repetition Crisis

February 17, 2016

View as Single Page
The Replication Crisis and the Repetition Crisis

With data becoming ever more abundant, this should be the golden age of the social sciences. And yet they seem to be suffering two mirror-image nervous breakdowns—the Replication Crisis and the Repetition Crisis.

Outright made-up-data fraud is hardly unknown in academia, but the double disasters have more to do with shortcomings in how contemporary researchers analyze relatively honest data. I suspect that the systemic failures stem more from researchers being allowed both too many and too few of that evocative (if actually rather dry) technical term: “degrees of freedom.”

One cause of the Replication Crisis has been that analysts grant themselves excessive post hoc liberties to crunch the numbers however many ways it takes to find something—anything—that is “statistically significant” (which isn’t the same as actually significant) and thus qualifies as a paper for publish-or-perish purposes. Hence, social scientists seem to be coming up with a surplus of implausible junk science findings on trivial topics, such as “priming” (the contemporary version of subliminal advertising), which then routinely fail to replicate.

In contrast, in what I’ll dub the Repetition Crisis (a.k.a. the Explanation Crisis), academics hamstring the interest and usefulness of their findings by ruling out ahead of time any explanatory factors other than the same tiny number of politically correct concepts that were exhausted decades ago.

“In 2016, blaming white privilege for everything you don’t like isn’t quite as lame as blaming the Bavarian Illuminati, but the gap is closing.”

Why a Repetition Crisis? Dissident social psychologist Jonathan Haidt of NYU’s Stern School of Business, author of The Righteous Mind, pointed out in a freewheeling interview with John Leo how the ever-growing list of sacred cows in American life restricts what social scientists can allow themselves to discover about important issues:

For many years now, there have been six sacred groups. You know, the big three are African-Americans, women and LGBT. That’s where most of the action is. Then there are three other groups: Latinos, Native Americans…and people with disabilities. So those are the six that have been there for a while. But now we have a seventh—Muslims.

One could argue that there are more sacred groups than seven, but Haidt’s next point was illuminating:

Something like 70 or 75 percent of America is now in a protected group. This is a disaster for social science because social science is really hard to begin with. And now you have to try to explain social problems without saying anything that casts any blame on any member of a protected group. And not just moral blame, but causal blame. None of these groups can have done anything that led to their victimization or marginalization.

For example, in discussing crime or poverty, social scientists are allowed to imply that the dirt that white people live upon is inherently magic while the dirt under black people is obviously tragic. But anything smarter and more interesting could get them furiously denounced by angry know-nothing students (or Watsoned out of their jobs if they lack tenure). So it’s safest just to blame everything and anything on white people.

Still, as the generations roll by, that’s increasingly sounding like a senile conspiracy theory. In 2016, blaming white privilege for everything you don’t like isn’t quite as lame as blaming the Bavarian Illuminati, but the gap is closing.

As the range of acceptable insights narrows, boredom stalks the social sciences.

Haidt notes:

Anthropology and sociology are the worst—those fields seem to be really hostile and rejecting toward people who aren’t devoted to social justice.

Today, for example, it seems astonishing that 60 years ago cultural anthropologists like Margaret Mead could be celebrities. The educated public now assumes that cultural anthropologists are pedantic and petulant, best avoided.

It’s not surprising, therefore, that many social scientists try to sidestep the Repetition Crisis by avoiding important issues in favor of marketing-research-like problems, which in turn worsens the Replication Crisis. (The central distinction between science and marketing research is that the latter doesn’t strive to discover permanent truths: That, say, Bill Cosby was good at advertising Jell-O Pudding Pops in 1979 is good enough for marketing research. If you want to know whether to hire Cosby in 2016, marketing researchers would be happy to take your money.)

One cause of the Replication Crisis is the social-science version of the Hollywood excuse “We’ll fix it in post.” As postproduction computer-generated imagery has gotten cheaper, movie directors have become more likely to rationalize on-set flaws in dialogue, acting, or their own direction with the reassurance that the scene can always be salvaged in postproduction by computer wizardry.