AMA Journal of Ethics^®

Case

Delta Health was one of three large health insurers that shared a Midwest multistate market. Recently all individual and group practices and clinics that were among Delta’s preferred provider organizations (PPOs) received letters from the company informing them that Delta was about to perform a quality audit on some providers, in the hope that the objectively acquired and reported information would help the physicians maintain or improve the quality of care. Delta employees would be calling to schedule appointments and reporting on the ease of getting a timely appointment, the helpfulness of the telephone and office staff, the physician’s attentiveness and response to their reasons for the visit, and the treatment recommendations. The results concerning a physician or clinic would be shared with that physician or clinic only.

Reaction to the Delta Health letter at one PPO, Mid-West Internal Medicine Clinic, was typical. At a specially called meeting, 12 of the clinic’s 15 physicians met to discuss what steps they should take to prepare for Delta audits if, indeed, they were among the “providers” visited.

“This is a secret shopper attack,” said the first speaker, “and we’re not Wal-Mart, and our patients aren’t customers. We have difficulty seeing everyone who really needs care in a timely manner. Now that patient is going to have to wait while we see a secret shopper who’s not even sick. It makes no sense.”

“I’m not worried in the least,” said a second physician. “It’s a one-time thing. They’re not going to send these fake patients in week after week. And we could get some observations that would really help. I don’t have a clue what patients expect when they first walk into the waiting room.”

But others feared that what Delta Health called “information to help maintain or improve quality care” was really going to be economic profiling. A physician whose husband had been getting treated at the faculty practice organization associated with an academic health center in a neighboring state said that Delta had recently dropped that organization as one of its PPOs.

Commentary

A physician’s central goal is to provide competent, compassionate care to her patients [1]. This principle of beneficence is paired with that of nonmaleficence: not only must a physician seek to benefit her patients, but she must also avoid harming them [2]. “Market forces, societal pressures, and administrative exigencies must not compromise this principle [of primacy of patient welfare],” states the American Board of Internal Medicine’s “Charter on Medical Professionalism” [3]. Yet in a world of rapidly evolving medical science, evidence-based medicine, and complex health care payment systems, providing care is a challenging task. Every day physicians must balance cutting-edge technology with accessible treatments, clinical guidelines with gut feelings, patients in need with the clinic’s bottom line. Certainly the Mid-West Internal Medicine Clinic would like to “maintain or improve quality care,” the stated goal of this planned evaluation. It would be naive, however, to assume that the motivations of insurance companies coincide in all respects with those of medical practices. A covert economic profiling effort could have unjust and damaging effects for the clinic. It would be cynical, however, to dismiss this planned evaluation as a Trojan horse. Using the “Charter on Medical Professionalism” as an ethical compass, I will examine the plan cautiously, but in good faith.

Both the Mid-West Internal Medicine Clinic and Delta Health share the goal of patient satisfaction. And the clinic, like any clinic, has room for improvement. “Physicians must be dedicated to continuous improvement in quality of health care” [3], states the charter. Thus the Mid-West physicians should welcome the opportunity offered by an objective evaluation—if that is what Delta Health truly intends. Examining several aspects of the planned evaluation, I will assess whether the use of secret shoppers in a medical setting is ethical, whether the data they gather are valid, if those data could be useful, and if the planned evaluation is fair. For several reasons, I believe that Delta Health’s planned evaluation should be opposed. I will suggest an alternative approach that would be more ethical and yield more useful results.

Is the Use of Secret Shopper Patients Ethical?

There are three obvious ethical critiques of the secret shopper method of evaluation: (1) secret shoppers take time and resources away from real patients; (2) the secret shopper method introduces deceit into the trusting patient-doctor relationship; and (3) the method violates the privacy of the physician’s office. None of these three critiques is convincing.

The first suggests that secret shoppers violate the principle of nonmaleficence, but it overestimates the scale of the evaluation and underestimates the capacity of a primary care practice to balance high demand with adequate care for all. Presumably, the Mid-West Clinic both accepts new patients and sees regular patients for follow-up. Seeing new patients—even secret shoppers—should not change the care of regulars. If it would, the clinic should not take new patients. At the Mid-West Clinic, the risk seems to be less about compromising the care of existing patients than about wasting their time. One physician whose clinic was evaluated by secret shoppers describes such criticism as “short-sighted,” arguing, “This is a miniscule amount of time and it can help you to serve your clients [patients] better in the long term” [4]. In a bustling primary care practice, the presence of a few secret shoppers would not materially change the quality or timeliness of care.

The critique that such methods introduce deception into the patient-doctor encounter discounts the importance of assuring clinical excellence, a goal of the evaluation. “The traditional patient-physician relationship requires that both parties be open and honest,” argues one physician [4]. While physicians are ethically bound to be honest with their patients [3], patients regularly lie to their doctors. They fib about diet, exaggerate symptoms to get sick notes, and fake pain to obtain narcotics. Doctors are trained to appreciate that things might not be what they first appear, but that they should nonetheless behave in a professional manner to all patients. Dishonesty may be present already in the patient-doctor relationship, but is it ethical for a payer to orchestrate a physician encounter that will certainly include deceit? There are precedents; concealment is introduced into the doctor-patient relationship during experiments to avoid bias [5]. This is justified for the good of advancing clinical knowledge. In this case, too, concealing the identity of the secret shoppers is critical to an accurate assessment. Surely the goal of improving quality justifies the methods of the secret shoppers.

The third ethical critique, that the secret shopper method violates privacy and is no better than snooping [6], ignores the fact that there must be checks on the patient-physician relationship. That encounter is intimate; physicians are ethically bound to protect their patients’ confidentiality [3]. Patients, however, are free to disclose information from that encounter to anyone they choose. They can fill out surveys, rate doctors on the Internet, or launch a public malpractice suit. The truth is that the encounter between a doctor and patient is only as private as the patient wants it to be. There is nothing new—or ethically questionable—about doctors being scrutinized and evaluated. The question of what happens to those secret shopper evaluations is discussed below, but the method does not violate the privacy of the exam room.

Do Secret Shopper Patients Gather Valid Data?

The use of secret shoppers to monitor quality is a relatively new development in the medical field, though they are used extensively in other industries [5]. Some argue that observations gathered by secret shoppers are merely a subjective snapshot that cannot be generalized to represent a physician or a practice [7]. That may be true, but the fact is that many aspects of a patient’s medical experience are difficult to assess. Patient surveys suffer from recall bias, and physicians tend not to respond to surveys at all [5]. Secret shoppers are trained to be systematic and dispassionate. Thus their evaluations may offer valid observations about the patient experience that are otherwise difficult to capture.

Delta Health states that their secret shoppers would comment on the availability of timely appointments, the helpfulness of clinic staff, the physician’s attentiveness, and her recommended treatment. The first three goals pertain to patient perceptions, and several published accounts of secret shopper evaluations suggest that they are well qualified to comment on this aspect of the medical experience [8-10]. The fourth goal is problematic. Clinical decision making is a complex process that cannot be fully evaluated with a checklist. Physicians should be expected to approach each patient in a professional manner—and can thus be evaluated at this level—but assessment of the treatment plan is beyond the scope of a one-size-fits-all exercise.

Would the Data Be Useful to Both Mid-West Internal Medicine Clinic and Delta Health?

Critics of the secret shopper model focus on the potential misuses of the data gathered. They claim that the evaluations might be used against the medical practice, in malpractice lawsuits or as a form of economic profiling [11]. Delta Health presents the plan as a nonpunitive, private quality improvement assessment. If this were truly the case, it could be useful to the clinic. Delta Health’s motivations, however, are open to question.

Many primary care practices have found secret shopper evaluations to be a helpful tool in improving the quality of their patients’ experience. One academic outpatient center recently published data on its experience with secret shoppers. Using their feedback, the center increased customer service scores, decreased wait times, and increased the size of the patient panel [8]. Another published study on the use of secret shoppers found that they offered valid observations on telephone triage that led to improvements in the system [9]. While some accounts raise concerns—for example, use of the data gathered by secret shoppers to fire employees—these reports are anecdotal [4, 7]. At present, it seems the best use of this new evaluation methodology is for a good-faith, nonpunitive quality assessment. With that stipulation, a secret shopper evaluation focusing on patient experience—and not the treatment plan—could be potentially useful to the Mid-West Internal Medicine Clinic.

Why this evaluation would be useful to Delta Health is unclear, particularly if, as stated, the company intends to share the results only with the evaluated physician or clinic. True, it is in Delta Health’s best interest to have excellent clinics in its preferred provider network. But a well-conducted secret shopper study is expensive, and Delta Health has no way of assuring that the clinics will make this investment worthwhile by acting on the secret shoppers’ feedback. Such information might, however, provide the company with data that could be used for economic profiling. Delta Health’s stated motivations for supporting such an evaluation are suspect because the study, as planned, would not certainly benefit the company.

Is the Proposed Evaluation Fair?

In certain situations, the secret shopper method offers an ethical, valid, and potentially useful way to evaluate patients’ perceptions of a medical clinic. It is impossible to achieve quality improvement without the consent and cooperation of the clinic staff, and Delta Health announced this evaluation by fiat. The plan also lacks transparency. Delta Health states that the data would be “objectively acquired,” but this is in some sense impossible since the secret shoppers will be company employees.

The planned evaluation is unfair. There is no guarantee that inappropriate evaluation factors would not be used, that the evaluations would be truly private, and that the data gathered would not be used in punitive ways. Finally, the clinic has been given no choice in the matter.

A Counter-Proposal

I suggest that the Mid-West Internal Medicine clinic write a letter to Delta Health stating that the clinic shares the goal of quality improvement but opposes this planned evaluation. The clinic would be willing, however, to experiment with the secret shoppers if Delta Health hires a third party to conduct that evaluation. The clinic would have an active role in determining which features the secret shoppers would evaluate. Given the relative newness of this method of quality evaluation, the clinic would use the data gathered in a nonpunitive way—it might make changes in how it does business, but would not fire any employees on the basis of this evaluation. Finally, to guarantee that that this evaluation is truly for quality improvement and not for economic profiling, the results would not be shared with Delta Health. The letter would raise the question of the company’s motivations in launching the study, since the plan has no mechanism to guarantee that the clinic acts on the quality evaluation, Delta Health should not need to see the data. The clinic’s willingness to work with a third-party evaluation company shows its dedication to quality improvement.

In this complex medical world, doctors must focus on the primacy of patient welfare. They must strive to help and to avoid harm. Central to the ancient art of healing is the act of listening. Though quality audits, such as the secret shoppers, may complement those conversations, nothing can replace a meaningful exchange between a patient and physician. Dr. Amy Friedman describes her own experience as a pseudo-secret shopper when she becomes a patient herself [12]. During the procedure, she feels “dispassionately processed rather than embraced.” From this experience she concludes, “Finding a moment to hold your patient’s hand, to look directly into her eyes, to ask about a grandchild or to even remember his name must retain importance, amidst the endless policies, financially coercive forces, and regulatory pressures.” Friedman reminds us that quality improvement begins with acts of compassion.