AMA Journal of Ethics^®

Berlin L. Accuracy of diagnostic procedures: has it improved over the past five decades? AJR Am J Roentgenol. 2007;188(5):1173-1178.

Research continues to indicate that medical practice is beset with a high rate of error and significant variations in diagnoses among physicians. In "Accuracy of Diagnostic Procedures: Has it Improved Over the Past Five Decades?" Leonard Berlin argues that the medical community has been aware of the high rate of errors and clinical variations for about 50 years, a claim that he supports by discussing the groundbreaking research on diagnostic and radiologic errors conducted by L. Henry Garland in the 1950s [1]. Garland 's 1959 article, the basis for Berlin 's discussion, was the first to reveal the high percentage of physician inaccuracy in a variety of diagnostic procedures. Berlin argues that, though research over the past half century has confirmed Garland 's conclusions, the medical profession has been unable (or unwilling) to make the changes necessary to decrease the rate of error or incidence of variation among diagnosticians [1]. In light of mounting evidence and technological advances, this record causes us to ask why the profession has not taken steps to improve diagnostic accuracy.

Early Research

Garland's research on the accuracy of radiologic and other diagnostic procedures revealed a "surprising degree of inaccuracy" including "a 34% error rate in the diagnosis of myocardial infarction," an "agreement rate of only 7% among five experienced pediatricians" in the diagnosis of malnutrition in children, and "a 28% error rate among clinical laboratories in measuring the erythrocyte count" [2]. Another of Garland 's striking findings was a 30 percent disagreement rate among experienced radiologists in diagnosing pulmonary tuberculosis [2]. When Garland's studies were published in the 1940s and '50s, the high rates of error and disagreement within the profession came as a shock to physicians and were described by a personal friend of the author's as "morally disturbing" [2, 3]. Berlin does not offer detailed discussion of Garland's explanation for the rate of error and variability among clinicians, simply noting that he attributed much of the error in radiologic diagnosis to basic human variation and the "still unexplained human equation" [2].

In the years since Garland 's research was published, other investigators have confirmed his findings. Studies of radiological technologies, such as the mammogram, have shown similarly high rates of inconsistencies [2]. Something not taken into account in this discussion of error rates is that, even though the proportion of errors remains the same, the increased use of many diagnostic procedures has benefited the overall health of more people. To use the same example as Berlin, the recommended guidelines for screening mammography and increased use of the technology by women have resulted in up to one-third more breast cancers being caught at an early stage than were detected at that stage before screening was recommended [4]. Probably, then, despite the consistent error rate, the findings from screening tests may have actually increased the number of people who benefit from these procedures.

Berlin looks closely at the ways in which error rate can be calculated and how different study methods affect the final report. The article explains how, if the denominator in the error rate calculation is the total number of patients screened, the error rate is bound to appear to be far lower than if the denominator is the number of patients who are positive for a given condition. In both cases the numerator is the sum of the missed positives and the false positives [2]. Moreover, some studies of medical error rates are not taken from the clinical arena but, for example, are conducted by giving a group of physicians a number of abnormal samples and calculating the error rate based on their performance [5]. The resulting error rates may serve well to demonstrate variability among physicians, but should not be interpreted as the error rate in clinical practice. Berlin notes that when the sample includes a mix of normal and abnormal cases, similar to what would be found in clinical practice, the error rate drops from an average of 30 percent to just 3.5-4 percent [5].

Though missing a positive finding will more than likely result, at the very least, in a delay of treatment for a patient, there does not seem to be an established link between those misses and the degree of harm caused to patients as a result. Berlin admits that this is an important shortcoming in his discussion of error rate and variability in diagnosis [5]. Furthermore there is no discussion of the rate at which inaccurate diagnoses are corrected early enough that the error is clinically insignificant. Both of these questions, though difficult to answer, are critical areas for exploration and study.

Berlin does not examine the ethical obligations of the medical profession when it comes to preventing error. What, for example, is an acceptable rate of inaccuracy? When should patients be told of variation in interpretation? What should be done if physicians agree that delay in diagnosis caused physiologic harm? What, if anything, is our responsibility to patients who receive false-positive reports and undergo further procedures unnecessarily? Should professional guidelines or practice standards recommend two or more readings of films?

In addition to patient-level outcomes, considerations of diagnostic errors in the health care system as a whole are missing from Berlin 's analysis. The financial burden and resulting possible resources wasted on patients who receive false-positive results as well as the possible increase in the cost of care for those whose treatment is delayed (and whose disease progressed) because they initially received false-negative results are well worthy of future study. The ethical issue of access to limited medical resources and care is something that adversely affects all health care consumers. The 1999 Institute of Medicine (IOM) report "To Err Is Human: Building A Safer Health System," estimated that medical errors cost the United States approximately $37.6 billion each year [6]. This estimate included all errors—not just diagnostic—and the whole spectrum of medical errors including adverse drug reactions, equipment failure, infection, surgical error, and misinterpretation of medical orders [7], must be addressed. In a previously published article, Berlin criticized the IOM report for the light in which it cast physicians, while at the same time noting appreciatively the increase in federal funding for programs like the Patient Safety Center that the report provoked [8].

More information on the specific financial burden of errors in diagnostic procedures will only help to clarify the magnitude of this problem and, as so much of our health care system is cost driven, possibly lead stakeholders throughout the system to work to decrease these types of errors. The medical profession is committed to reducing error and improving patient safety. Opinion 8.121, "Ethical Responsibility to Study Error and Prevent Harm," which first appeared in the American Medical Association's Code of Medical Ethics in 2003, guides physicians to "strive to ensure patient safety" and notes that they should "play a central role in identifying, reducing, and preventing health care errors" [9].

Can New Research and Technology Help?

It is too soon to know whether the advent of new radiological technologies including digital radiography and computer-aided detection (CAD) are helping to improve accuracy in diagnosis. CAD increases sensitivity and decreases variability of interpretation and has been shown to be of value for physicians, but Berlin seems unsure of whether, after decades of improvements in data and technology, CAD will be the golden ticket to decreasing an error rate that has remained unchanged for so long [5]. The research Berlin cites has been published within the past two years, so it is possible that the profession is still learning about and adjusting to these tools.

Several times in his conclusion Berlin returns to the truism that there will always be the element of human error to contend with [5,10]. He fails to mention the efforts by an entirely separate field of medical research to examine why "perceptual and cognitive errors occur and what steps can be taken to ameliorate them" [10]. One specific aspect of cognition being explored is the precise capability of the human visual system with regard to displayed radiologic monitor images. The goal here is to present information in an easier-to-comprehend format and thus improve diagnostic performance [11].

Though the importance of Garland 's findings and his effort to highlight degrees of disagreement and error in diagnostic medicine cannot be discounted, Berlin 's views on emerging diagnostic technologies as well as the current efforts being made to reduce error seem overly pessimistic. The still-emerging and yet-to-emerge improvements in digital imaging, along with efforts to understand how human vision can best read and learn, are important steps being taken to fulfill the profession's ethical responsibilities to reduce or prevent error.