In the Literature
Dec 2010
Alasbali T, Smith M, Geffen N, et al. Discrepancy between results and abstract conclusions in industry- vs. nonindustry-funded studies comparing topical prostaglandins. Am J Ophthalmol. 2009;147(1):33-38.
At the end of a busy clinic day a drug representative sits you down to discuss her product. She produces a glossy advertisement highlighting the drug’s efficacy with various graphs and references. She then provides a copy of a peer-reviewed article that compares her product to a competitor’s. Without even reading the article you can guess what the conclusions will be—that the rep’s product is better—but to confirm, you skim the abstract and, as suspected, the abstract conclusions state that this company’s product performed better than the competitor’s. You also notice that the study was funded by your visitor’s company. A week earlier, a representative from the competing company made a similar visit, but in that case the peer-reviewed abstract concluded that their product was superior. You look for that article and find that the study was funded by the competitor.
In medicine, we are trained to base decisions on evidence. In deciding which medication to prescribe, for example, there are a variety of considerations: efficacy, side effects, contraindications, ease of use, and cost, among others. Comparing medications from different classes often makes decisions regarding first-line treatment relatively easy; however, when a group of medications produce a given clinical result by means of the same mechanism, the decision may not be as clear. Decisions should be evidence-based, but can we trust the evidence? And which evidence is most trustworthy?
In 1997, Pharmacia introduced a novel medication, latanoprost, for the treatment of glaucoma. As a result of this medicine’s superior efficacy, daily administration, and relatively small number of side effects, latanoprost quickly became the first-choice drug for treating glaucoma—in industry terms, a “blockbuster.” This fact did not go unnoticed by the pharmaceutical competition, which subsequently developed drugs with a similar mechanism of action (“me-too” products), including travoprost (Alcon) and bimatoprost (Allergan). The end result is a variety of products with similar mechanisms of action competing for the same market share. Manufacturers of competing drugs fund studies of their respective products both to create opportunities for clinicians to gain experience with their products and to contribute to the peer-reviewed literature. Their ultimate objective is to influence the decisions of prescribers. Does industry funding of these studies bias results?
In hopes of answering this question, we searched the literature up to November 2007 for peer-reviewed articles on studies that compared the ocular hypotensive efficacy of topical prostaglandins. We reviewed each article to determine whether the findings for the main outcome measures were statistically significant and then compared the statistical findings in the results section to the language in the abstract conclusions. The publications were also divided up into industry-funded and non-industry-funded categories, the results of which were compared.
There were 39 eligible publications, of which roughly 75 percent reported on industry-funded research. In those industry-funded-research articles, there was agreement between the results of the main outcome measure and conclusions in the abstract 38 percent of the time, but in 62 percent of publications, there was disagreement between the results of the main outcome measure and the abstract conclusions. Among the nonindustry-funded studies, the agreement between the results of the main outcome measure and the abstract conclusions was 100 percent (p=0.0006). In publications of the industry-funded research, only 24 percent found a statistically significant main outcome measure, but 90 percent had pro-product conclusions [1].
You may wonder about the high percentage of pro-manufacturer conclusions in the cases where the main outcome results were not significant. Such conclusions result from authors’ emphasis on surrogate outcomes, minimizing nonsignificant main outcomes and reporting numerical rather than statistically significant differences. Emphasizing secondary outcomes and ignoring the results of the main outcome measure is a process referred to as “spin.” Some studies look at multiple comparisons in attempts to find an outcome of statistical significance. This is referred to as “data dredging.”
These findings are certainly not unique to ophthalmology or publications on prostaglandins. A recent report of published randomized controlled trials with nonsignificant main outcome measures found evidence of spin in 58.3 percent of the abstracts [2]. In addition, there are many studies in the literature outlining the effect of industry bias in publications involving a variety of diseases and therapies [3-19].
Physicians must be aware that it is essential to read literature carefully. One cannot rely solely on the abstract but should read papers in full, focusing on methodology and reading the complete results, because the prevalence of spin is greater in abstracts than in the main text [2]. It is equally important to read the disclosures to determine if a potential for bias exists.
In our study, 4 of the 39 publications did not include published disclosures. By contacting the authors directly, we discovered that 3 of these publications were industry-funded [1]. The International Committee of Medical Journal Editors policy states that “all participants in the peer review and publication process must disclose all relationships that could be viewed as presenting a potential conflict of interest” [20]. In June 2007, when we reviewed the disclosure policies of all indexed English-language ophthalmology journals, we were able to find published web-based policies for authors in 79 percent, for peer reviewers in 7 percent, and for editors in 5 percent of the journals. After we contacted the editors directly, these results changed to 100 percent for authors, 60 percent for peer reviewers, and 33 percent for editors (not counting 5 journals that did not respond to the question regarding authors and 12 journals that did not respond to the question regarding peer reviewers and editors) [21].
Going forward, measures need to be taken to minimize the possibility of industry bias. Although some have suggested that industry should be prohibited from funding studies, this is unlikely to occur, especially in view of decreased funding by government and nonprofit organizations. Currently, authors and editors have significant responsibility for upholding the integrity of published literature, and perhaps stronger policies are required both for disclosure of potential conflicts and presentation of results. Ultimately, however, it is readers who must take time to read articles in full, and not simply rely on the abstract, before drawing their own conclusions.
References
- Alasbali T, Smith M, Geffen N, et al. Discrepancy between results and abstract conclusions in industry- vs. nonindustry-funded studies comparing topical prostaglandins. Am J Ophthalmol. 2009;147(1):33-38.
- Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058-2064.
- Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):454-465.
- Davidson RA. Source of funding and outcome of clinical trials. J Gen Intern Med. 1986;1(3):155-158.
- Rochon PA, Gurwitz JH, Simms RW, et al. A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. Arch Intern Med. 1994;154(2):157-163.
- Cho MK, Bero LA. The quality of drug studies published in symposium proceedings. Ann Intern Med. 1996;124(5):485-489.
- Friedberg M, Saffran B, Stinson TJ, Nelson W, Bennett CL. Evaluation of conflict of interest in economic analyses of new drugs used in oncology. JAMA. 1999;282(15):1453-1457.
- Djulbegovic B, Lacevic M, Cantor A, et al. The uncertainty principle and industry-sponsored research. Lancet. 2000;356(9230):635-638.
-
Clifford TJ, Barrowman NJ, Moher D. Funding source, trial outcome and reporting quality: are they related? Results of a pilot study. BMC Health Serv Res. 2002;2(1):18.
-
Kjaergard LL, Als-Nielsen B. Association between competing interests and authors’ conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ. 2002;325(7358):249.
-
Baker CB, Johnsrud MT, Crismon ML, Rosenheck RA, Woods SW. Quantitative analysis of sponsorship bias in economic studies of antidepressants. Br J Psychiatry. 2003;183:498-506.
- Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326(7400):1167-1170.
- Leopold SS, Warme WJ, Fritz Braunlich E, Shott S. Association between funding source and study outcome in orthopaedic research. Clin Orthop Relat Res. 2003;415293-301.
-
Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials. A reflection of treatment effect or adverse events? JAMA. 2003;290(7):921-928.
- Bhandari M, Busse JW, Jackowski D, et al. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ. 2004;170(4):477-480.
- Finucane TE, Boult CE. Association of funding and findings of pharmaceutical research at a meeting of a medical professional society. Am J Med. 2004;117(11):842-845.
- Shah RV, Albert TJ, Bruegel-Sanchez V, Vaccaro AR, Hilibrand AS, Grauer JN. Industry support and correlation to study outcome for papers published in Spine. Spine (Phila PA 1976). 2005;30(9):1099-1104.
- Perlis RH, Perlis CS, Wu Y, Hwang C, Joseph M, Nierenberg AA. Industry sponsorship and financial conflict of interest in the reporting of clinical trials in psychiatry. Am J Psychiatry. 2005;162(10):1957-1960.
- Yank V, Rennie D, Bero LA. Financial ties and concordance between results and conclusions in meta-analysis: retrospective cohort study. BMJ. 2007;335(7631):1202-1205.
-
International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals: ethical considerations in the conduct and reporting of research: conflicts of interest. http://www.icmje.org/ethical_4conflicts.html. Accessed August 23, 2010.
- Anraku A, Jin YP, Trope GE, Buys YM. Survey of conflict-of-interest disclosure policies of ophthalmology journals. Ophthalmology. 2009;116(6):1093-1096.
