AMA Journal of Ethics^®

Case

A large hospital system, wanting to improve outcomes on coronary artery bypass surgeries (CABG), has decided to implement a public surgery report card for cardiothoracic surgeons based on morbidity and mortality outcomes on these procedures. While sitting in the doctors’ lounge, Dr. Smith, a cardiologist, overhears her colleagues discussing the report card system.

“You have to refer your patients to the best possible cardiothoracic surgeon. If I know that a surgeon has high complication rates, why would I risk the health of my patient?” explains Dr. Cummings.

Dr. Thomas looks skeptical. “So you think those reports will be accurate? Do you really think the cases will be risk adjusted so that the surgeons who take the most difficult cases don’t end up getting the worst ‘grades,’ so to speak? And what happens when one of your patients needs an emergency CABG and the best surgeon is unavailable? Are you going to advise him or her to refuse the surgery?”

Dr. Smith hears her pager and gets up to return the page. The call center connects her with the hospitalist on service to notify her that Mr. Green—a long-time patient—has just been admitted and is on his way to the catheterization lab. The hospitalist notes that he is planning to consult cardiothoracic surgery if Mr. Green’s catheterization report reveals clogged arteries and the need for CABG surgery. Ordinarily, Dr. Smith would have referred the patient to Dr. Anderson, who has performed CABG surgery for most of her patients. But now she considers what she overheard in the lounge and replies, “Let me call you back with a specific surgeon to consult. I need to look something up.”

Commentary

One of the ways the Patient Protection and Affordable Care Act (ACA) has transformed American health care is through a strong push for transparency, stimulating the development and public dissemination of health care report cards so that patients, physicians, and health care organizations may be better informed about their choices of clinician. These evaluations rely on either process measures or outcomes measures. Process measures, or “quality indicators,” describe activities performed by professionals and staff, such as completion of immunizations or screening tests. Outcomes measures, on the other hand, assess the results of the health care intervention the patient has undergone and may include complication rates, length of hospital stay, mortality and morbidity rates, and patient satisfaction with care [1]. Our discussion focuses on physician report cards (PRCs) based on patients’ treatment outcomes and the value of these report cards for clinicians. PRCs’ comparative outcome data may help both new and seasoned physicians in their referrals and may be a useful tool for self-assessment and improvement. Report cards, however, must be taken as merely one tool in the referral toolbox.

Benefits of PRCs for Physicians

Physician report cards assist patients and physicians in choosing clinicians with track records of high-quality care [1]. Traditionally, physicians have relied on word of mouth from colleagues or patients, as well as patients’ personal experiences, to identify the best physicians available. As a quantitative way to discriminate among physicians, PRCs may give referring physicians the knowledge to make this decision themselves. Sometimes, physicians may receive financial and non-financial “kickbacks” for referrals to a specific physician or medical practice, which PRCs may help deter. PRCs may also assist physicians new to a hospital or region in directing patients to the highest-quality physicians. This is especially vital if they do not have the benefit of word of mouth from colleagues or their own patients’ experiences. Recent graduates from residency, still early in their careers, may greatly benefit from these evaluations.

Whether PRCs are made publicly available or used internally within a practice or hospital, objective and comparative measurements of performance also help physicians understand their own professional strengths and weaknesses and encourage them to improve [2]. This, in turn, is meant to promote competition among physicians to provide high-quality care [1, 3]. Hospitals can use PRCs to assess their physicians for new or continued medical staff privileges and to better understand deficiencies or variation in hospital standard procedures [4].

Although the general public does not rely on publicly available PRCs as much as expected [5], and some experts argue that there is not convincing evidence that PRCs are beneficial [6, 7], some studies have found that publicly available report cards may lead to improved outcomes [8]. For instance, Werner et al. found that the quality of nursing home postacute care improved with the release of PRCs, although rehospitalization rates did not change [9]. In a study based on the New York state coronary artery bypass surgery report-card system, patients who visited the surgeons with the highest report card ratings were roughly half as likely to die as those who visited bottom-quartile surgeons [10]. The same study found that, with each release of publicly available PRCs in New York, surgeons performing coronary artery bypass grafts (CABG) who fell into the bottom quartile of performers were more likely than their higher-performing counterparts to move clinical locations or switch careers.

Limitations of Report Cards

Inconsistent availability. Despite their potential benefits, PRCs have many limitations, one being that completed PRCs may not be representative of the physician population. Most PRCs are voluntary, meaning that low-performing or skeptical physicians may opt out of evaluations [1]. Health care insurers and state hospital associations also tend to produce more PRCs than state medical associations [11]. More primary care physicians are evaluated than specialists, and more report cards are produced to evaluate hospital performance or practices than individual physicians [11].

Difficulty of use. With no standardized system for PRCs, hospitals and practices have great leeway in choosing which metrics to use and how to word findings. Thus, physicians and patients may have difficulty comparing PRCs from different systems.

While it would be comforting to think that PRCs would reduce unethical referral practices (e.g., kickbacks for referral to a specific physician or practice), they may, indeed, be ignored. Even with comprehensive PRCs, physicians may be skeptical about how measures are collected and refer their patients to physicians within their institutions, perhaps out of familiarity or loyalty [12]. As it stands, physicians, for the most part, do not rely heavily on PRCs [6, 13, 14], and hospitals tend to care about them primarily for purposes of quality improvement and cost management [5, 11]. PRCs are not yet used by most of the medical community and are not part of the “standard of care.”

Inaccuracy. Physician report cards may not accurately reflect a physician’s performance. If they are not risk adjusted to account for care of sicker patients, physicians who treat higher-risk patients may have less favorable PRCs than those who turn such patients away [3]. Taking PRCs at face value may mean referring to a physician whose score does not accurately reflect his or her level of skill or who is likely to turn away a “risky” patient to preserve or boost performance marks [15-19].

Even if PRCs are risk adjusted, there are ways for physicians to game the system. In an effort to boost or preserve his or her PRC score, a physician may document that the patient is healthier following treatment than he or she actually is or reduce the amount of pain reported by a patient [16].

Even if PRCs are risk adjusted and physicians participate honestly, differences in outcome measures between a high- or low-scoring physician and the group average often are not statistically significant, and sample sizes for individual physicians tend to be small.

Inapplicable results. The physicians with the “best” report cards simply may not be readily available. Perhaps a long queue of patients precludes them from tending to an emergency, or their schedules are booked well in advance. Moreover, these physicians may be too geographically distant from patients for a visit to be practical [13]. Some patients—those with less pressing health problems, more accessible transportation, or even personal connections within the medical community—may have better access to the most-highly rated physicians.

Questionable relevance. A given physician’s PRC may not rate outcomes for a specific medical condition [3, 15], even a common one, or may disproportionately report on outcomes for one disease over another [11]. Similarly, the PRC may be a general evaluation, rating only morbidity and mortality outcomes and access. And event rates are typically low (e.g., mortality may happen in only 1 to 3 percent of cases), at least for CABG [20], raising the question of whether physicians’ performance differences even matter. Also, although the medical profession has placed a great emphasis on developing physicians’ communication and interpersonal skills [21], PRCs often do not measure these [22].

Some have suggested including in PRCs “measures of the appropriateness of care” so that physicians do not provide low-risk patients unnecessary treatments more suitable for high-risk patients [1], a practice that would lower their rates of complications and mortality.

Ethical Considerations in Our Case

The four basic principles of medical ethics—respect for autonomy, beneficence, nonmaleficence, and justice—factor into the decision to use PRCs. Although a physician may choose not to disclose using a PRC in making a choice for referral, Brown et al. argue that informing patients about how and why their physicians may use PRCs demonstrates a respect for informed consent and patient autonomy [12]. With knowledge of predicted risk and return associated with a specific treatment from a particular physician, the patient has necessary information about a treatment, in keeping with the physician’s responsibility to both benefit a patient and keep him or her out of harm’s way [23], and remains free to refuse it [24]. But if the quality of that information is compromised or subpar, it is difficult to know how to uphold the principle of nonmaleficence, which requires that physicians “not intentionally create a harm or injury to the patient, either through acts of commission or omission” [23]. A physician may inadvertently run afoul of the principle of nonmaleficence by making a patient referral based on a PRC drawn from dubious or incomplete information. Knowingly referring a patient to a doctor who has an overinflated PRC score, engages in “downcoding” patients’ actual posttreatment medical problems, or avoids treating patients from particular demographic groups would certainly violate the principle of nonmaleficence. On the other hand, PRCs provide some objective measures and have—to a degree—been shown to help improve patient outcomes [8, 9], so neglecting available PRCs may constitute a failure to promote the principle of beneficence, the responsibility to benefit a patient and improve his or her health [23]. Because they are not yet in the standard of care, however, failing to act on a PRC is not a neglect of beneficence. Although not readingavailable PRCs is unethical, ignoring evaluations because of certain mitigating factors is ethical.

Finally, the principle of justice—“a fair distribution of goods in society” [23]—is the least likely to be upheld when using PRCs. As discussed previously, existing health and social disparities may contribute to a given patient’s “riskiness,” and a physician may deem a particular ethnicity [17] or socioeconomic group [18] at greater risk for worse outcomes and refuse to treat patients in these groups.

Handled correctly, PRCs can provide a valuable tool for physicians referring patients. Until PRCs become ubiquitous, a physician in Dr. Samuelson’s position should check PRCs but also be allowed to rely on her own, colleagues,’ or patients’ opinions of the preferred physician. The decision is easy if opinion and the PRC lead to the same conclusion. However, if the PRC is incongruent with popular opinion, Dr. Samuelson should, beyond reading evaluations, consider whether:

1. the PRC has a sufficiently large sample size, reports statistically significant differences in outcomes, and has measures pertinent to a patient’s specific condition;
2. the report accounts for patient-specific risk (if not, skepticism is reasonable);
3. the subject physician is likely to turn away a potentially risky patient or may have subconscious biases regarding ethnicity and socioeconomic status;
4. the PRC includes measurements of interpersonal or communication skills;
5. the “best” physician for a referral can readily see a patient and is geographically accessible.

Undoubtedly, following all of these suggestions can help maximize PRCs’ utility while ensuring that they do not inadvertently harm patient health. However, following these steps for every patient in every referral situation is highly unrealistic. Especially because this may be a time-sensitive situation and the best-rated surgeon may not be available, it is ethically permissible for Dr. Samuelson to make her referral in good faith to a less highly ranked physician if the most highly ranked is not available. In this case, she must be allowed to rely most heavily on word of mouth of colleagues or patients, as well as personal experience.

Conclusion

PRCs are increasing in availability. They may provide both new and seasoned physicians with comparative metrics to help guide referral patterns and may be a useful tool for self-assessment and improvement. They also may have a role in moving towards a value-based care model that accounts not just for the quantity of services provided but also for their quality [25]. However, PRCs have major limitations. They frequently contain inconsistent measures of outcomes, address certain outcomes disproportionately, lack adequately large sample sizes or statistically significant differences in patient outcomes, and, sometimes, do not adjust for patient risk. A physician desiring to consult a PRC in referring a patient to a colleague may find that his or her institution does not even produce them. Even when a PRC exists and contains accurate information pertinent to an individual patient’s needs, the most highly rated physician may not be available or may be too geographically distant. For these reasons, PRCs are not yet the standard of care for physician referrals, but Dr. Smith ought to use her judgment if accurate PRCs are available and pertinent in her patient’s case.