Journalistic deficit disorder

[Van!]

http://www.economist.com/node/21563275

Reporting science

Journalistic deficit disorder

What newspapers don’t say matters as much as what they do

Sep 22nd 2012 | from the print edition

[1] IF ALL the stories in the newspapers claiming that a cure for cancer is just around the corner were true, the dread disease would have been history long ago. Sadly, it isn’t. But though some publications do have a well-deserved reputation for exaggeration in this area, many of these reports are at least based on respectable research published in peer-reviewed journals. So what is going on?

[2] That, more or less, is the question addressed by François Gonon of the University of Bordeaux and his colleagues in a paper in the Public Library of Science. Dr Gonon, a neurobiologist, looked not at cancer but at another disease that gets a lot of coverage in the press, attention-deficit hyperactivity disorder (ADHD)—a fashionable diagnosis for children who struggle to pay attention at school.

[3] Dr Gonon’s group scoured PubMed and Factiva, two databases containing biomedical papers and newspaper articles respectively, for reports on ADHD. Comparing the two turned up 47 papers on ADHD that were covered in 347 articles written in English-language newspapers during the 1990s. From these, the researchers chose to focus on the ten papers that had received the most press coverage.

[4] First, they studied subsequent scientific literature to see what had become of the claims reported in the top ten papers. Seven of these had reported research designed to test novel hypotheses. Though each concluded at the time that the hypothesis in question might be correct (ie, the data collected did not refute it), the conclusions of six were either completely refuted or substantially weakened by the subsequent investigations unearthed by Dr Gonon. The seventh has neither been confirmed nor rejected, but he and his colleagues, citing two independent experts on ADHD, say its hypothesis “appears unlikely”.

[5] The other three papers in the top ten were following up existing hypotheses rather than presenting novel ideas. Two of them were confirmed by the subsequent work Dr Gonon tracked down, whereas one was weakened.

[6] All this is to be expected. Research on research—particularly on medical research, where sample sizes are often small—shows that lots of conclusions do not stand the test of time. The refutation of plausible hypotheses is the way that science progresses. The problem was in the way the work was reported in the press.

Absence of evidence

[7] In total, the original top ten papers received 223 write-ups in the news. But then the newspapers lost interest. Dr Gonon and his team found 67 further studies examining the conclusions of the original ten, but these subsequent investigations earned just 57 newspaper articles between them. Moreover, the bulk of this coverage concerned just two of the ten. Follow-ups to the other eight got almost no attention. Dr Gonon’s team do not pull their punches. There is, they say, an “almost complete amnesia in the newspaper coverage of biomedical findings”.

[8] If the hypotheses reported in the original studies had stood the test of time, then such short journalistic attention spans would not be a problem. But as Dr Gonon has shown, 80% of the papers in the study turned out to be either wrong or questionable. Failure to follow that up risks leaving even attentive readers with a false impression of progress in the field.

[9] It would be easy to point the finger at lazy journalists for this state of affairs, and a journalistic version of attention-deficit disorder is, no doubt, partly to blame, for the press has a natural bias towards the new and exciting. But science itself must carry some responsibility, too. Eight of the ten articles whose fates Dr Gonon studied were published in respected outlets such as the New England Journal of Medicine and the Lancet. The deflating follow-ups, by contrast, languished in more obscure publications, which hard-pressed hacks and quacks alike are less likely to read.

[10] A sensible prescription is hard. The matter goes beyond simply not believing what you read in the newspapers. Rather, it is a question of remembering that if you do not read subsequent confirmation, then the original conclusion may have fallen by the wayside. And, for what it is worth, as The Economist went to press, a search on Google News suggested that, a week after its publication, not a single newspaper had reported Dr Gonon’s paper.

from the print edition | Science and technology

[lumiel]

흔히 한국 특유의 국민성이라고 말하는 '냄비근성'이 사실 영미권에서도 찾아볼 수 있는 거였군요 ㅎㅎ 역시 사람 사는 건 비슷한가봐요~

[lumiel]

그리고 무엇보다 주의력 부족 및 과잉 행동 장애 (ADHD)를 a "fashionable" diagnosis for children who struggle to pay attention at school 라고 표현한 부분이 흥미롭네요 :^D

[Van!]

의견감사함다 ^^

[로빈]

고맙습니다.