Monday, January 9, 2012

The Chump Effect

Andrew Ferguson on the gullibility of journalism majors, many of whom have never taken an advanced mathematics or science course in their life, when it comes to uncritically accepting any 'scientific studies' results announced, especially if they confirm the reporters biases.

Lots of cultural writing these days, in books and magazines and newspapers, relies on the so-called Chump Effect. The Effect is defined by its discoverer, me, as the eagerness of laymen and journalists to swallow whole the claims made by social scientists. Entire journalistic enterprises, whole books from cover to cover, would simply collapse into dust if even a smidgen of skepticism were summoned whenever we read that “scientists say” or “a new study finds” or “research shows” or “data suggest.” Most such claims of social science, we would soon find, fall into one of three categories: the trivial, the dubious, or the flatly untrue.

A rather extreme example of this third option emerged last month when an internationally renowned social psychologist, Diederik Stapel of Tilburg University in the Netherlands, was proved to be a fraud. No jokes, please: This social psychologist is a fraud in the literal, perhaps criminal, and not merely figurative, sense. An investigative committee concluded that Stapel had falsified data in at least “several dozen” of the nearly 150 papers he had published in his extremely prolific career.

Perhaps “falsified” is too mild a word. Stapel didn’t just tweak and twist numbers, he made stuff up. With his colleagues, Science Insider reported, “he would discuss in detail experimental designs, including drafting questionnaires, and would then claim to conduct the experiments at high schools and universities with which he had special arrangements. The experiments, however, never took place.” Questionnaires are the mother’s milk of social science, given (most often) to collections of students who are easily accessible to the scientist. After being rewarded with course credits or money, the students go on to serve as proxies for humanity in general, as the scientist draws from their questionnaires large conclusions about the way human nature compels us, all of us, to think and act.

The conclusions that Stapel drew were large indeed. One thing he liked to demonstrate in his studies was the exploitive nature of democratic capitalism. Last year, the New York Times reported on a typical Stapel study, called “The Self-Activation Effect of Advertisements.” It proved that advertising for cosmetics and fancy shoes “makes women feel worse about themselves,” as the Times put it. Another study, released at the end of the scandal-ridden year 2009, was called “Power Increases Hypocrisy.” Quite a timely little study it was. Stapel and his colleagues’ research revealed that powerful people were more likely to be “moral hypocrites.” And which powerful people did the researchers have in mind? “Politicians [who] use public funds for private benefits while calling for smaller government” and CEOs “accepting executive bonuses while simultaneously asking for government bailouts.”

Both of these studies purported to employ the usual social-psychology method: Students in psychology or marketing classes were asked to “role-play” or perform some artificial task under the observation of graduate students. Then they’d fill out those questionnaires to report their thoughts or feelings.

Sometimes, though, social psychologists move beyond the lab. A good example is a more recent study from Stapel’s corpus, released last spring to wide publicity. It touched on another of Stapel’s favorite themes: white racism.

“Disorder can encourage stereotyping, study says,” read the headline in the Los Angeles Times. Stapel discovered—scientifically, of course—that white heterosexuals used racism and homophobia as defense mechanisms. Confronted with disorder in their “social environment,” Stapel showed, they quickly reverted to their natural inclination to stereotype “the other” and draw comfort from their prejudice.

At this writing, investigators are not yet clear to what extent the results of these particular studies are discredited by Stapel’s fakery. And nobody knows how extreme an anomaly Stapel’s behavior will prove to be. Leslie John of Harvard Business School recently surveyed more than 2,000 social psychologists about their research methods. She found a rash of research practices she deemed “questionable.” Indeed, she wrote, in social psychology, “some questionable practices may constitute the prevailing research norm.”

But it hardly seems to matter, does it? The silliness of social psychology doesn’t lie in its questionable research practices but in the research practices that no one thinks to question. The most common working premise of social-psychology research is far-fetched all by itself: The behavior of a statistically insignificant, self-selected number of college students or high schoolers filling out questionnaires and role-playing in a psych lab can reveal scientifically valid truths about human behavior.

And when the research reaches beyond the classroom, it becomes sillier still.

The rest here.

No comments: