{"id":89934,"date":"2015-06-05T20:30:29","date_gmt":"2015-06-05T18:30:29","guid":{"rendered":"http:\/\/businesstech.co.za\/news\/?p=89934"},"modified":"2015-06-05T18:29:03","modified_gmt":"2015-06-05T16:29:03","slug":"bad-science-and-snappy-headlines-will-make-you-believe-anything","status":"publish","type":"post","link":"https:\/\/businesstech.co.za\/news\/media\/89934\/bad-science-and-snappy-headlines-will-make-you-believe-anything\/","title":{"rendered":"Bad science and snappy headlines will make you believe anything"},"content":{"rendered":"

Earlier in the year the world was finally treated to some good news from science: a report was published that claimed to show that eating chocolate could help you lose weight faster<\/a>.<\/p>\n

Although it all seemed too good to be true, the story was reported in news outlets around the world. Europe\u2019s largest daily newspaper, Bild, ran it on the front page. It made TV news in Australia and the US<\/a>, it landed on the Irish Examiner<\/a>, The Times of India<\/a>, and the Huffington Post in various<\/a> languages<\/a>.<\/p>\n

But it was<\/em> too good to be true. Or, if you\u2019re an aficionado of the work of trolls, it was even better.<\/p>\n

Last week science journalist John Bohannon revealed<\/a> that the whole study was an elaborate prank, a piece of terrible science he and documentary film makers Peter Onneken and Diana L\u00f6bl \u2013 with general practitioner Gunter Frank and financial analyst Alex Droste-Haars \u2013 had set up to reveal the corruption at the heart of the \u201cdiet research-media complex\u201d.<\/p>\n

Terrible science<\/h3>\n

So what did they do? Bohannon and his team went through all the standard practices of science. But at every stage they chose methods they knew would lead not to truth, but to clickbaity headlines.<\/p>\n

\"Has<\/a>

Has the Daily Star gone coco? Screenshot by author<\/p><\/div>\n

To begin the study they recruited a tiny sample of 15 people willing to go on a diet for three weeks. They divided the sample into three groups: one followed a low carbohydrate diet; another followed that diet but also got a 42 gram bar of chocolate every day; and finally the control group were asked to make no changes to their regular diet.<\/p>\n

Throughout the experiment the researchers measured the participants in 18 different ways, including their weight, cholesterol, sodium, blood protein levels, their sleep quality and their general well being.<\/p>\n

And here\u2019s their first trick. Measuring such a tiny sample in so many ways means you\u2019re almost bound to find something vaguely reportable. As Bohannon explains it<\/a>:<\/p>\n

Think of the measurements as lottery tickets. Each one has a small chance of paying off in the form of a \u201csignificant\u201d result that we can spin a story around and sell to the media. The more tickets you buy, the more likely you are to win. We didn\u2019t know exactly what would pan out \u2014 the headline could have been that chocolate improves sleep or lowers blood pressure \u2014 but we knew our chances of getting at least one \u201cstatistically significant\u201d result were pretty good.<\/p><\/blockquote>\n

And so then they submitted it for publication. But again, Bohannon chose the path that led away from truth, picking a journal from his extensive list of open access academic journals<\/a> (more on this below).<\/p>\n

Although the journal, (International Archives of Medicine<\/a>), looks somewhat like a real academic journal, there was no peer review<\/a>. It was accepted within 24 hours, and published two weeks later.<\/p>\n

But great publicity!<\/h3>\n

Practiced in the white magic of science journalism and familiar with the dark arts of science PR, Bohannon then whipped up a press release<\/a> he knew would bait the world\u2019s media.<\/p>\n

\"Hmm,<\/a>

Hmm, chocolate study? Screenshot by author<\/p><\/div>\n

The key, Bohannon stated, was to \u201cexploit journalists\u2019 incredible laziness\u201d \u2013 to write the press release so that reporters had the story laid out on a plate for them, as it were.<\/p>\n

As he later wrote, he \u201cfelt a queazy mixture of pride and disgust as our lure zinged out into the world\u201d. And a great many swallowed it whole.<\/p>\n

Headlines around the world screamed Has the world gone coco? Eating chocolate can help you LOSE weight<\/a>, Need a \u2018sweeter\u2019 way to lose weight? Eat chocolates!<\/a> and, perhaps more boringly, Study: Chocolate helps weight loss<\/a>.<\/p>\n

Some of these reports remain online today in the same state as they were published, although some outlets, such as Cosmopolitan Germany<\/a>and Huffington Post India<\/a>, have since updated to reveal the sting. The Australian TV news piece has been deleted<\/a>, like the mistake never happened.<\/p>\n

What\u2019s the washup?<\/h3>\n

The reporters around the world who cut-and-pasted Bohannon\u2019s press release certainly aren\u2019t blameless. None did the due diligence \u2013 such as looking at the journal, looking for details about the number of study participants, or even looking for the institute<\/a> Bohannon claimed to work for (which exists only as a website) \u2013 that was necessary to find out if the study was legitimate.<\/p>\n

But if we\u2019re really looking to find fault here, we\u2019ve got to cast our net a bit wider. As Bohannon and his colleagues noted, there\u2019s a \u201cdiet research-media complex\u201d here that\u2019s almost rotten to the core.<\/p>\n

From beginning to end we\u2019ve got a system with almost as much scope for corruption as a BBQ at a high ranking FIFA official\u2019s house:<\/p>\n