How Social Scientists, and the Rest of Us, Got Seduced By a Good Story


#1

How Social Scientists, and the Rest of Us, Got Seduced By a Good Story
by Megan McArdle


Apr 30, 2013 3:34 PM EDT

Almost two years ago, the field of social psychology was rocked by some astounding news: Diederik Stapel, one of its stars, had been faking his research. I don’t mean that he’d been subtly altering figures to give better results, or maybe running through a series of increasingly implausible modeling assumptions until they delivered the results he’d expected. I mean he’d apparently given up doing experiments entirely. Instead he imagined experiments, imagined which results would look good, and then sat down at a computer and entered those numbers into a spreadsheet.

The New York Times Magazine has an incredible article this week describing what Stapel did, and how he did it. What’s less clear is why he did it, or how he was able to get away with faking results for seven long years. …

While Stapel’s parents are wrong to excuse his cheating, they are right to some extent: the system produced this screwed up result, and Jonah Lehrer, and many of the other fraudsters who have taken us in over the years.

The Times piece strongly suggests that the field of social psychology was leaving the doors wide open and a “Welcome, Burglars!” mat on the front porch. That’s all too easy to believe if you’ve read Nobel Laureate Daniel Kahneman’s scathing open letter to some of his colleagues about the sloppiness of their research practices. Well accepted effects are turning out to be hard to replicate outside of the labs of the people who discovered them. Social Science research is vulnerable to all manner of statistical shenanigans, and a number of academics seem to be exploiting those vulnerabilities, either accidentally or deliberately.

This news story captures reasons why many are skeptical of “sciences” such as psychology and sociology. Their subjects, humans, are very complex, and those doing the studies come into their work with preconceived ideas. In this story those come together in almost the worst way: the data from this researcher’s work was too inconsistent/complex to support any conclusion; the researcher responded by cooking his data and simply making stuff up. Worse, he entangled and/or involved doctoral students he was supervising or otherwise working with in his fraud. As McArdle points out, briefly, Stapel is not unique, just recent and one who has admitted what he did rather than obfuscate or blame-shift (he’d never make it as a politician!).

At the same time, McArdle is right that “the system” to some degree encourages what Stapel did. Professors are under professional and financial pressure to produce: do research projects and publish. Universities trust their researchers, and the vaunted “peer review” process won’t catch a skilled fraudster in a field like psychology. Much of the general public views researchers with a sort of awe. And academics are in continual circle-the-wagons mode against philistine-skeptics. Guess what? As this case shows, the “philistines” are right sometimes!


#2

I have been skeptical of psychology and sociology for not adhering to sound and reliable processes wrt how they perform their studies. I became accutely aware of it when I took senior level psychology course as an elective on college. Most of the experiments and studies discussed were interesting but made wide sweeping conclusions based upon very small and skewed samples with little or no follow-on studies to determine repeatability. If I were to have conducted the many experiements, studies, and analyses that I have performed over the last 30 years in that manner, I would have not graduated from college or fired from my job for incompetence.


#3

IMO in psychology the problem is not with the researchers themselves, the limitations on the conclusions seem to be stated clearly in any studies I read, the problem is with the media distorting or reporting the conclusions without the limitations to make them seem sensational, and people who just read the abstract… but then again my interest is limited to the biological perspective of psychology where there is much less room for subjectivity.

Sociological fields are another story, many sociologists have believed since at least the 70s that they have an ethical obligation to aid a side in any political issue they are studying (of course this ends up being the subordinate side).
The primacy of the ethical: Propositions for a militant anthropology


#4

Oh you have no idea…Human Subject Review Board…


#5

The problem with psychology and sociology is that there does seem to be a sort of old boys network or paradigm that must be adhered to in order to get grant money and hold tenure. Right now that paradigm is mostly a pro materialism favored one. It changes every few hundred years, but yes scientists have been known to either skew the data or read it in a favorable light towards their worldview.

Science isn’t immune from human biases.