AMA Journal of Ethics^®

Safety and reduction of error have traditionally been important issues in fields such as the airline industry; more recently, safety has become a priority issue in health care. This increased interest in safety and error reduction in medicine has been due in no small measure to the Institute of Medicine's groundbreaking report, To Err is Human: Building a Safer Health System (IOM Report).¹ Many articles discussing error prevention strategies cite the IOM Report, particularly the statistic that 44 000 to 98 000 people die every year as a result of medical error.² The IOM Report was widely noted in the lay press as well as in the medical community; even Oprah Winfrey devoted a special episode of her famous talk show to the issue.³ This focused attention has made patient safety and error reduction priority issues in health care.

The 44 000 to 98 000 preventable death figures are an extrapolation of data reported in other studies. A review of these studies is important if one is to analyze the IOM Report fairly. The report is clear that preexisting data were used to underscore the urgent need to reduce medical error and that it does not offer any new data on the frequency and impact of medical errors.

The report explores and discusses the relevant literature and research and has an excellent table summarizing its sources.⁴ Unfortunately, not everyone who cites the report has read the entire document, and it is frequently misunderstood as a "study" that "demonstrated" the incidence of preventable deaths attributable to medical errors. Instead of being a study, the IOM Report is actually a policy document that discusses the scope of medical errors and makes recommendations to improve patient safety. Anyone who wishes to be active in safety improvement and error reduction in medicine must understand the report's contents and conclusions and be able to apply this information competently. This particularly applies to members of the media.

The IOM Report analyzes the scope and nature of medical errors by offering a comprehensive analysis of the existing data on the impact of errors on patient safety. It then proceeds to make recommendations for improving safety in the existing health system.⁴ The impact of medical errors on national mortality rates is a crucial component of the report's foundation. Two studies are cited that looked at the impact of medical error on patient mortality. These data are meaningful, but each study has limitations. Both studies were huge undertakings, and the researchers' ability to analyze data was compromised by the magnitude of the patient pools. As with any critical analysis of a body of research, it is important to identify the structure, definitions, data collection strategy, subject base, and researcher information to analyze and apply the results.

The first study discussed in the report used data from New York collected in 1984 and then reported in 1991.⁵ Data in the other study were collected in 1992 in Utah and Colorado and published in 2000.⁶ The New York study, known as the Harvard Medical Practice Study, reviewed 30 121 randomly selected charts for adverse events. It defined an adverse event as "an injury that was caused by medical management (rather than the underlying disease) and that prolonged hospitalization, produced disability at the time of discharge, or both."⁴ The statewide incidence of adverse events was estimated to be 3.7 percent, of which 1.0 percent was due to negligence. When these numbers were applied to the number of statewide discharges, using a weighting procedure described in the article, there were 98 609 adverse events in 1984 in New York State, 27 179 of which were due to negligence. It was estimated that 13 451 patients died "at least in part as a result of adverse events," and 13.6 percent of all adverse events led to death.

The study performed in Utah and Colorado reported results similar to those of the Harvard Medical Practice Study.⁴ This study used the same definition of an adverse event, but the reviewer training and quality control in the chart review process were different. Roughly 2.5 percent of all discharges were randomly sampled and reviewed for adverse events. Adverse events occurred at a rate of 2.9 percent. Death resulted in 8.8 percent of adverse events due to negligence. The total proportion of adverse events causing death was 6.6 percent.

The 2 studies found relatively similar overall rates of adverse events, but suggested that different percentages of adverse events resulted in death. The reasons for these differences are discussed in both the Utah/Colorado study and the IOM Report.^1,2,3,4 The IOM Report then used the 2 rates of death due to adverse events reported in the studies and extrapolated this to the total number of US hospital admissions in 1997. The total number of estimated admissions was 33.6 million. When the results of the New York study are applied (13.6 percent of adverse events leading to death) the number of deaths due to adverse events was 98 000 for the entire United States in 1997. When the Utah/Colorado results are used (6.6 percent of adverse events leading to death) the number of deaths in the United States in 1997 is estimated to be 44 000. This is the claim seized by the media—that 44 000 to 98 000 people die each year due to medical errors, making medical errors the 8th leading cause of death in the United States.²

Although these figures are frequently invoked in both the medical and lay literature, some commentators have expressed criticism at the way these original studies arrived at the now-famous figures. Authors from the Regenstrief Institute at Indiana University stated in JAMA:

Both were observational studies and were not designed to describe causal relationships. The Harvard study authors included caveats, such as "lead [sic] to death" and "died at least in part as a result of adverse event." The authors of the Colorado-Utah study reported a proportion of patients who died in the adverse reaction group, but said nothing about the cause of these deaths. The IOM did not mention any of these limitations in its report.⁷

Troyen Brennan, one of the investigators in the New York study, makes the point even clearer when he states:

Perhaps more to the point, neither study cited by the IOM as the source of data on the incidence of injuries due to medical care involved judgments by the physicians reviewing medical records about whether the injuries were caused by errors. Indeed, there is no evidence that such judgments can be made reliably.⁸

Both comments make clear that the original data used by the IOM Report had some serious limitations. Yet, few media commentators have publicized these limitations, focusing more on the very high figures cited by the report (especially the higher 98 000 figure).

One of the few media figures who has commented on the misuse of the Report by members of the media is Susan Dentzer, health care correspondent for "The Jim Lehrer Newshour." Dentzer has criticized news journalists for focusing on the high numbers, giving them a "misleadingly totemic significance," as well as inaccurately equating errors with acts of medical malpractice and neglecting to focus on the system issues behind many errors.⁹ Dentzer lays most of the blame with number-hungry journalists who often defer to the authority of statistics. Dentzer also asserts, however, that the IOM Report itself contributed to this number craze with the following assertion in its executive summary: "More people die in a given year as a result of medical errors than from motor vehicle accidents (43 458), breast cancer (42 297), or AIDS (16 516)."⁹ This type of comparison with stark numbers obviously makes good copy for most print journalists.

Journalists such as Dentzer have played an important role in highlighting the misuse of reports with tempting statistics. Unfortunately, her piece was written in an obscure medical journal that does not reach out to a mass audience. Considering that most consumers and patients receive so much of their information about health care through the media, it behooves journalists to report more carefully on the contents of reports such as the IOM's To Err is Human.