Virtual Mentor. March 2007, Volume 9, Number 3: 188-192.

Journal Discussion

• e-mail |
• print |
• view pdf

The Media Miss Key Points in Scientific Reporting

Media coverage of information presented at medical meetings often fails to qualify the findings reported, and scientists and the media need to develop a better working relationship to ensure the accuracy of early-stage research reports.

Namarta Kotwani

Woloshin S, Schwartz LM. Media reporting on research presented at scientific meetings: more caution needed. Med J Aust. 2006;184:576-580.

The lay public’s understanding of medical science and its perceptions about recent advances in research are primarily mediated by mainstream news sources. Unfortunately, media coverage of science and health news is often sensationalized, inaccurate and dumbed down for the masses. Since most of us have a voracious appetite for definitive news about medical discoveries and cures and care little for the arcane facts and methodologies that characterize the scientific process, the media sometimes spin bland stories to capitalize on our thirst for the latest miracle cure or diet. Health news can also be affected by the relative inability of reporters to evaluate specialized research studies. Carried away by the headline-making potential of preliminary findings put forth at influential medical association meetings, journalists may be tempted to present works in progress as definitive breakthroughs. For these reasons, medical news is particularly vulnerable to distortion.

In 2006, Steven Woloshin and Lisa M. Schwartz published a systematic evaluation of the media coverage of scientific meetings in The Medical Journal of Australia [1]. The authors sought to discover whether media stories about research presented at major North American scientific meetings reported basic study facts, cautiously interpreted results and highlighted the preliminary nature of the work presented. This study’s results illustrate how preliminary research is misrepresented in the popular media and show that even slight modifications in the phrasing of medical news could go far toward ensuring that the public received a more nuanced perspective on current medical research.

The study
Based on advice from science writers, editors and media database searches, the authors chose five high-profile meetings that were likely to receive media attention. The selected meetings were the annual sessions of the American Heart Association, the International AIDS Conference, the American Society of Clinical Oncology, the Society for Neuroscience and the Radiological Society of North America. Two major media databases, LexisNexis and ProQuest, provided archived media stories that appeared within two months of each meeting in 2002-2003. The authors found 210 newspaper stories and 20 nationally syndicated television or radio transcripts from the U.S. and Canada pertaining to these five meetings. They included all the news stories that reported on a single research presentation and stories that reported on multiple presentations, if at least one of the presentations related to the story’s headline. Stories that superficially commented on multiple presentations at a conference were excluded. Ultimately, the authors analyzed 174 newspaper stories and 13 television and radio transcripts.

The authors applied an explicit coding scheme to the analysis of each news story to determine whether the reporter included basic study facts, provided relevant cautions about study design and indicated the preliminary nature of research. Basic study facts included size, subjects (animals, cells, humans), design (random, controlled) and main results of the study. Relevant cautions included caveats about the study design and its “intrinsic limitations.” Was the study conducted on animals? Was its sample size too small to yield reliable conclusions? Was the study uncontrolled? Was it controlled but not randomized? The coders also noted whether the public was informed of the preliminary nature of research. Did the reporters indicate if the presentations at the meetings were associated with an in-press or published peer-reviewed article? Was the public warned that the presentation featured ongoing work and that results were likely to change as the study matured?

Schwartz and another physician, blinded to the study objectives, served as coders. Both were trained in clinical epidemiology and analyzed each news story individually. The coders were in “almost perfect” agreement about the analysis of each item. Woloshin independently coded all items for which there was disagreement and established the final codes.

Results
The authors concluded that news stories about scientific research meetings often omit basic study facts and cautions. They found that 89 percent of the news stories identified studies as being conducted on live humans and 9 percent as animal or lab studies; subjects could not be identified in 2 percent of media reports. Of the 187 stories, 34 percent failed to mention study size, and 35 percent reported it so ambiguously that it could not be confidently determined by expert readers. Forty percent of the stories quantified the main result. Only 6 percent of those that discussed animal studies mentioned their limited applicability to humans. Of those that reported on studies with small sample sizes only 21 percent noted problems with the generalizability of the findings. A mere 10 percent of stories about uncontrolled studies suggested that it was not possible to know whether outcomes really corresponded to exposure, and 19 percent of reports on controlled but not randomized studies mentioned that other confounding factors could be responsible for the differences between the control group and treatment group. Only 29 percent of 142 news reports on interventions mentioned risks or potential downsides. Twelve stories noted a corresponding in-press article in a medical journal, and of the remaining 175, only two suggested that the results were unpublished, likely to change or not peer reviewed [2].

The Woloshin and Schwartz article emphasized that work presented at scientific meetings is not ready for public consumption and that less media coverage of such preliminary research is warranted. Yet, as the authors recognize, “too many interests are served by turning preliminary reports into health news” [3]. Researchers and their academic affiliates benefit from media attention because publicity attracts patients and donors. The meeting itself is in the limelight; media coverage ensures more advertising and encourages other high-profile scientists to attend future meetings. The authors urged meeting organizers and scientists to issue explicit, modest and nuanced press releases and statements to offset sensationalized media reports.

Discussion

Media coverage of preliminary research can influence clinical practice. For instance, results from Cancer and Leukemia Group B (CALGB) Study 9344 were presented at the American Society of Clinical Oncology Annual Meeting in May 1998 and widely disseminated through popular media outlets [4, 5]. Following the meeting and the media coverage, the use of taxanes rose dramatically as part of the chemotherapy regimen for primary breast cancer in the United States. The FDA, however, did not grant approval for the use of paclitaxel—a drug in the taxane category—in early-stage breast cancer until October 1999, and the data from the trial were not published in a peer-reviewed journal until 2003. As the case of paclitaxel demonstrates, the oral presentation of a single study at a large, well-publicized conference can accelerate a drug’s use in clinical settings. Although there was considerable skepticism from the scientific community about exposing women to toxic agents whose benefits were not clearly established, women with early stage breast cancer did benefit from paclitaxel.

The paclitaxel experience represents the best-case scenario. In contrast, Iressa, a drug developed for the treatment of patients with non-small-cell lung cancer who did not benefit from prior chemotherapy was approved by the FDA in 2003 on the basis of a preliminary, uncontrolled study [5]. In 2004, a placebo-controlled clinical trial showed that Iressa failed to prolong the lives of people with advanced lung cancer, but by then the drug had been prescribed to more than 200,000 patients worldwide [6]. Further, Iressa was approved despite concerns about major side effects observed among Japanese patients [5]. As early as 1999, the FDA had approved Taxotere, an effective alternative second-line therapy for non-small-cell lung cancer, which could have been given to these patients with a greater chance of success. The Iressa case demonstrates that the rapid dissemination of non-peer-reviewed preliminary claims is enormously risky because trial data can be prematurely (or erroneously) applied.

Misleading media reports can also cause patients to question appropriate standard therapies and pin their hopes on unrealistic treatment plans. In May 1998, the New York Times featured a front-page story on two anti-angiogenesis compounds, endostatin and angiostatin, which showed promise in treating tumors and cancers in animals [7]. Peppered with optimistic predictions by prominent scientists, including Nobel Prize winner James Watson and Richard Klausner, who was then the director of the National Cancer Institute, the article resulted in a flurry of calls to U.S. cancer clinics [7]. Desperate patients wanted access to these two cancer “drugs,” although the compounds had yet to be shown to be effective in humans. Some patients at Memorial Sloan-Kettering Cancer Center in New York City asked to wait for the new drugs rather than to proceed with standard chemotherapy.

Once an impression is made on the public’s mind, it can be hard to undo. In a 1998 television interview, Arpad Pusztai from the Rowett Research Institute in the U.K. declared that genetically modified potatoes expressed a lectin gene that was toxic to rats [8]. Pusztai’s results had not been corroborated by his peers, but the interview led to dozens of media scare-stories on dangers of “toxin-laden” genetically modified food. The following year the results of Pusztai’s research were published in the Lancet, which revealed that the conclusions presented during the television interview and later extrapolations were flawed. This questionable “evidence” further tarnished the image of genetically modified foods and agricultural biotechnology in Europe. In other instances, findings discussed at poster sessions, at talks at meetings or in informal discussions with colleagues can cause initial hysteria but may ultimately languish unpublished. As C. Neal Stewart writes:

The day a paper has passed peer review and is accepted for publication is the first time its content becomes a part of the scientific canon. Of course, scientists often discuss findings and disseminate them locally via the grapevine…but these information exchanges should not be viewed with the same significance or impact as publication of a formal paper in a peer-reviewed journal….It is the duty of both scientists and media to recognize this [8].

Although scientists must be cautious when interacting with the media, the fear of sensationalism should not lead to shirking exchanges. Hayes and Grossman observe that many scientists are reluctant to communicate with reporters because each media appearance poses a threat to their painstakingly acquired academic credentials [9]. One slapdash newspaper article or poorly edited TV appearance can undermine a reputation. Appropriate public visibility, however, can have several benefits for researchers. A scientist can share her special expertise by helping the public understand policy issues through the mainstream media. Media attention also highlights scientists’ institutional affiliates, attracting patients and alumni donations. A researcher with a positive media profile is more likely to connect with investors and industry and has better chances of developing a commercially viable product or obtaining a patent.

Federal and state funding of research and development in academic institutions amounts to about $19 billion and $13 billion, respectively [10]. Since government funding is so crucial for an academic researcher, remaining in the public eye allows her to demonstrate the utility and impact of her work to funding agencies. Funding priorities and legislation related to medical research can change, and regulators as well as proposal reviewers need evidence of the broader public impact of research to make scientist-friendly policies. Indeed, media attention can be quite rewarding for researchers.

Both scientists and reporters must ensure that media coverage of research findings is not compromised. While peer-reviewed articles have passed fairly rigorous standards of scrutiny, preliminary findings presented at conferences are prone to revision and radical correction. Scientists and the media need to develop a healthy working relationship to ensure that early-stage research presented at scientific meetings is reported in a manner that does not mislead the public. Boosting circulation or viewership, promoting individual careers, and drawing attention to one’s institutions at the public’s expense are reprehensible. As Woloshin and Schwartz suggest, press releases issued by meeting organizers, granting agencies and academic institutions should include balanced data presentations and study cautions. When researchers are interviewed at scientific meetings, they should indicate that their work is still awaiting peer review. It is not productive to stereotype the media as sensationalistic vultures. Scientists themselves are not always disinterested purists reluctant to talk to the media. Nor are they, at the other extreme, self-promoters desperate for a shot at instant celebrity. We must acknowledge our collective cultural tendency to hanker after the next nugget of juicy medical news and appreciate how much harm can be done if a misleading news story is seeded in the public imagination.

References

1. Woloshin S, Schwartz LM. Media reporting on research presented at scientific meetings: more caution needed. Med J Aust. 2006;184:576-580.
2. Woloshin S, Schwartz LM, 578-579.
3. Woloshin S, Schwartz LM, 580.
4. Giordano SH, Duan Z, Kuo YF, Hortobagyi GN, Freeman J, Goodwin JS. Impact of a scientific presentation on community treatment patterns for primary breast cancer. J Natl Cancer Inst. 2006;98:382-388.
5. Woloshin S, Schwartz LM. What's the rush? The dissemination and adoption of preliminary research results. J Natl Cancer Inst. 2006;98:372-373.
6. Pollack A. Lung-cancer drug shows unfavorable trial results. New York Times. December 20, 2004:C2.
7. Marshall E. The power of the front page of The New York Times. Science. 1998;280:996-997.
8. Stewart CN Jr. Press before paper—when media and science collide. Nat Biotechnol. 2003;21:353-354.
9. Hayes R, Grossman D. A Scientist's Guide to Talking With the Media: Practical Advice from the Union of Concerned Scientists. Piscataway, NJ: Rutgers University Press; 2006:xii.
10. Hayes R, Grossman D, 13.

Namrata Kotwani is a fellow in the Department of Clinical Bioethics at the National Institutes of Health in Bethesda, Maryland.