Thursday, June 15, 2017

Scientific and statistical ignorance in the media

I posted the comment below to this very fine post by Fran Porretto:

This, of course, is compounded by the almost complete scientific and statistical ignorance of the media. It's one of my largest peeves. After Trump pullled out of the Paris Accord, there was a video making the rounds of Arnold Schwarzenegger saying how bad this was and that air pollution killed 200,000 Americans every air. Ignoring, that the Paris Accords purpose was to control CO2, not air pollution, I wondered where the hell Arnold got this statistic about 200,000 people killed by air pollution, in the US, one of the countries with the cleanest air in the developed world and cleaner here than anytime in the past 2 centuries. A quick google search revealed dozens of headlines from 2013 stating basically that "MIT study says air pollution kills 200,000/yr". An popular alternate headline from the same study was that "More auto deaths are caused by auto emissions than accidents".

On it's face, it's a ridiculous assertion, but apparently reporters are innumerate enough to not even be curious. There are about 2.8m deaths in the US each year from all causes, so the assertion that almost 7% of deaths are 'caused' by air pollution should have raised some alarm bells. So given a couple of hours of free time, I decided to do what apparently none of the reporters who wrote the stories could be bothered to do (assuming they were competent enough to do so) and go and look at the actual research.

The MIT study which was the source of all this wasn't even actually a 'study' per se. Basically they took CDC/WHO estimates of the increased relative risk of people with existing pulminary and cardiac conditions to increases of, specifically, fine particulate pollution, designated PM2.5 (particulate matter < 2.5 microns). They also included ozone, but I will ignore that here. They then took EPA estimates of levels of PM2.5 pollution and combined the two to get their final estimates. Now looking at a couple of the orignal WHO studies, the error bars around the health effects were large and most likely underestimated. (If I have lung cancer, trying to tease out how much my life is shortened by increased exposure to PM2.5 concentrations is a pretty difficult thing to separate from dozens of other possible causes. EG PM2.5 tends to be higher in cities, living in which usually have higher stress levels as well, so the increase could well be from the increased stress). Even accepting their error bands, which were wide, and the error bands from EPA estimates of the actual levels of PM2.5 in various locations and various sources (which also had wide error bands), it is still questionable whether the MIT study calculated the error bands from combining the two correctly. Even so, those error bands were wide. So at 90% (not 95% as normal) the actual number was not 200,000 but 90,000 to 360,000 people affected significantly by air pollution. I assume they chose the 90% level because it may not have been significantly different from zero at the 95% level.

But taking all of the above and accepting the MIT study at face value, what it actually said was that 200,000 people with pulminary and cardiac conditions experienced reductions in the estimated lifespan by something on the order of 3-5% (ie if someone with lung cancer living in the pristine mountains of Arizona had an estimated lifespan of 10 years and moved to the nasty city of Los Angeles, they might expect a reduction of their lifespan on the order of 5% or about 6 months).

From that numerous 'news' sources (papers, TV, etc...) reported and I'm sure numerous people (like Arnold) believe that air pollution kills 200,000 people a year in the US alone. Just imagine the death toll in Beijing!

No comments: