Retraction Watch
Tracking retractions as a window into the scientific process
Conservative political beliefs not linked to psychotic traits, as study claimed
with 6 comments
Researchers have fixed a number of papers after mistakenly reporting that people who hold conservative political beliefs are more likely to exhibit traits associated with psychoticism, such as authoritarianism and tough-mindedness.
As one of the notices specifies, now it appears that liberal political beliefs are linked with psychoticism. That paper also swapped ideologies when reporting on people higher in neuroticism and social desirability (falsely claiming that you have socially desirable qualities); the original paper said those traits are linked with liberal beliefs, but they are more common among people with conservative values.
We’re not clear how much the corrections should inform our thinking about politics and personality traits, however, because it’s not clear from the paper how strongly those two are linked. The authors claim that the strength of the links are not important, as they do not affect the main conclusions of the papers — although some personality traits appear to correlate with political beliefs, one doesn’t cause the other, nor vice versa.
In total, three papers have been corrected by authors, and a correction has been submitted on one more.
We’ll start with an erratum that explains the backstory of the error in detail. It appears on “Correlation not Causation: The Relationship between Personality Traits and Political Ideologies,” published by the American Journal of Political Science:
The interpretation of the coding of the political attitude items in the descriptive and preliminary analyses portion of the manuscript was exactly reversed. Thus, where we indicated that higher scores in Table 1 (page 40) reflect a more conservative response, they actually reflect a more liberal response. Specifically, in the original manuscript, the descriptive analyses report that those higher in Eysenck’s psychoticism are more conservative, but they are actually more liberal; and where the original manuscript reports those higher in neuroticism and social desirability are more liberal, they are, in fact, more conservative.
The erratum (which is long, so we won’t quote in full) explains:
The potential for an error in our article initially was pointed out by Steven G. Ludeke and Stig H. R. Rasmussen in their manuscript, “(Mis)understanding the relationship between personality and sociopolitical attitudes.” We found the source of the error only after an investigation going back to the original copies of the data. The data for the current paper and an earlier paper (Verhulst, Hatemi and Martin (2010) “The nature of the relationship between personality traits and political attitudes.”Personality and Individual Differences 49:306–316) were collected through two independent studies by Lindon Eaves in the U.S. and Nichols Martin in Australia. Data collection began in the 1980’s and finished in the 1990’s. The questionnaires were designed in collaboration with one of the goals being to be compare and combine the data for specific analyses. The data were combined into a single data set in the 2000’s to achieve this goal. Data are extracted on a project-by-project basis, and we found that during the extraction for the personality and attitudes project, the specific codebook used for the project was developed in error.
In the paper, the authors are clear that “psychoticism” doesn’t mean “psychotic.” Rather:
Having a high Psychoticism score is not a diagnosis of being clinically psychotic or psychopathic. Rather, P is positively correlated with tough-mindedness, risk-taking, sensation-seeking, impulsivity, and authoritarianism…As such, we expect higher P scores to be related to more conservative political attitudes, particularly for militarism and social conservatism.
That 2012 paper has been cited 45 times, according to Thomson Reuters Web of Science.
Pete Hatemi, a political scientist at Penn State University and co-author on three of the papers, explained why the swapped political beliefs and personality traits do not affect the conclusions:
We only cared about the magnitude of the relationship and the source of it … None of our papers actually give a damn about whether it’s plus or minus.
When we asked Hatemi to elaborate on what that magnitude was — how much more likely were people who held conservative or liberal views to exhibit certain traits? — he said:
[T]he correlations are spurious, so the direction or even magnitude is not suitable to elaborate on at all- that’s the point of all our papers and the general findings.
Not all correlations were spurious to the authors, who reported on some in Table 1 of the paper. And the direction of the relationship seemed important enough that other researchers noticed it and prompted the authors to issue a correction. To help us make sense of the analysis, we turned to Andrew Gelman, a statistician at Columbia not involved with the work, to explain the AJPS paper to us. He said:
I don’t find this paper at all convincing, indeed I’m surprised it was accepted for publication by a leading political science journal. The causal analysis doesn’t make any sense to me, and some of the things they do are just bizarre, like declaring that correlations are “large enough for further consideration” if they are more than 0.2 for both sexes. Where does that come from? The whole thing is a mess.
He added:
It’s hard for me to care about reported effect sizes here…If the underlying analysis doesn’t make sense, who cares about the reported effect sizes?
We asked Steven Ludeke, who currently works at Colgate University University of Southern Denmark, how he first noticed the errors in the papers. He told us he found them while a student at the University of Minnesota:
At the time I was writing my preliminary written exam to qualify to be a PhD candidate, in which I had a section on “socially desirable responding” and sociopolitical attitudes. Because I’d recently read many other papers on the topic, once I came across the papers we’re discussing it is was immediately obvious to me that they’d reported their results wrong.
Brad Verhulst, a researcher at Virginia Commonwealth University and a co-author on all four papers, said that it’s unclear whether the error originated from the authors, or the group that conducted the surveys:
I don’t know where it happened, all I know is it happened. It’s our fault for not figuring it out before.
Once they pinned down the errors, Hatemi and Verhulst contacted all four journals to issue corrections; three have appeared so far.
The next paper that has been corrected is “Political Attitudes Develop Independently of Personality Traits” inPLOS ONE, which was flagged with a correction last summer:
There is an error in the third sentence of the fourth paragraph in “The Nature of Personality Traits and Political Attitudes” section of the Introduction. The correct sentence is: For the FFM and Eysenck’s personality traits, correlations between Openness and social attitudes, Neuroticism and economic attitudes, Conscientiousness and social attitudes, the P-Scale and military/defense attitudes, and Social Desirability and social attitudes are the most consistently found.
There is an error in the eighth sentence of the first paragraph in the Measures section of the Materials and Methods. The correct sentence is: Attitude factors were coded so that higher values reflect more liberal attitude positions.
That 2015 paper has been cited twice, once by the correction.
A correction also appeared last summer in Personality and Individual Differences for “The nature of the relationship between personality traits and political attitudes,” which has been cited 34 times since it was published in 2010. It explains the same error that the AJPS correction does, so we won’t quote from it here.
A correction is forthcoming for “Using genetic information to test causal relationships in cross-sectional data,” cited six times since it appeared in 2012, which describes the mathematical model behind the others. (Hatemi is not a co-author on that paper). The editor of the Journal of Theoretical Politics told us:
We have been in touch with Brad and are planning as of now to publish the correction in the July 2016 issue.
Even if the errors don’t affect the conclusions of the paper, they matter, Ludeke told us:
The erroneous results represented some of the larger correlations between personality and politics ever reported; they were reported and interpreted, repeatedly, in the wrong direction; and then cited at rates that are (for this field) extremely high. And the relationship between personality and politics is, as we note in the paper, quite a “hot” topic, with a large number of new papers appearing every year. So although the errors do not matter for the result that the authors (rightly) see as their most important, I obviously think the errors themselves matter quite a lot, especially for what it says about the scientific process both pre- and post-review.
Update, June 9th 10:30 am:
Ludeke informed us that his paper pointing out the errors is slated for the August issue of Personality and Individual Differences, and is now available online. It is titled, “Personality correlates of sociopolitical attitudes in the Big Five and Eysenckian models.”
Like Retraction Watch? Consider making a tax-deductible contribution to support our growth. You can also follow us on Twitter, like us on Facebook, add us to your RSS reader, sign up on our homepage for an email every time there’s a new post, or subscribe to our new daily digest. Click here to review our Comments Policy. For a sneak peek at what we’re working on, click here.
Posts
