Policy Forum
Dec 2003

A Critical Review of Standardized Patient Examinations as Part of the USMLE

Hillary Johnson
Virtual Mentor. 2003;5(12):572-576. doi: 10.1001/virtualmentor.2003.5.12.pfor1-0312.

 

Evidence that the poor communication skills of some older physicians contribute to medical malpractice lawsuits has led some groups to advocate for standardized clinical skills testing of all graduating US medical students. A new national exam for medical licensure plans to assess clinical skills using standardized patient exams. This is not an evidence-based intervention, however, because it targets the wrong population, and its efficacy has not been empirically validated.980s, an independent nonprofit corporation, the National Board of Medical Examiners (NBME), developed a standardized patient exam for certification of international medical graduates (IMGs) who wished to practice medicine in the United States. This was, in part, to prevent nonnative English speakers with poor communication skills from practicing in the United States. The exam became a requirement for IMGs in 1998 and cost $1200.

In recent years, the NBME has driven the call for a national exam assessing the clinical skills of US medical graduates for the purpose of medical licensure. According to the NBME, "the test will protect patient safety. A large body of literature has shown that poor communication skills and interpersonal and general clinical skills are related to a higher incidence of malpractice suits, lower treatment compliance by patients, and decreased patient satisfaction."1 The NBME and its sponsoring organization, the Federation of State Medical Boards (FSMB), jointly own and administer the standardized examinations that states require for medical licensure, the USMLE. Together, these organizations will begin requiring US medical school graduates to take the USMLE Step II CS (with a clinical skills component like that formerly given to IMGs) at 1 of 5 testing centers for a fee of $950. The test's format is similar to the standardized patient assessments currently being performed at most medical schools. It assesses competent data gathering, interpersonal skills, and patient notes. Passing the USMLE Step II CS will be a necessary step for all medical students in the class of 2005 and beyond to receive medical licensure.

The NBME's citation of the association of poor communication and interpersonal skills with a higher incidence of malpractice suits is accurate in part. Those physicians who are sued most often are frequently sued for perceived lapses in their communication skills or their perceived attitude rather than their proclivity for medical errors.2-5 But the NBME's claims that examining the clinical and communication skills of medical students in their final year will reduce errors and improve physicians' communication skills and patient safety are based on speculation. There is strong evidence that most poor outcomes in medicine are the result of systemic errors rather than an individual's errors.6-8 When clinical incompetence is the root of the error, often the physician is impaired or has been practice for more than 10 years and failed to continue his or her clinical training.9 Review of the American Medical Association (AMA) MasterFile physician database indicates that not 1 physician had been sanctioned by a medical board for communication skills within the first 5 years of clinical practice.10 Finally, while there is data in the literature regarding poor medical student and resident communication skills, there is no outcome data showing this exam will effectively screen these candidates from the physician pool.

In the process of medical licensing policy reform, medical student involvement is minimal to nonexistent. It is rare for state medical licensing boards to entertain a medical student presence. Although medical students are the NBME's largest customer base, they fill only 2 seats on the NBME's 80-member board. The 2 students are appointed by the American Medical Student Association and the Student National Medical Association. The NBME convenes a "reference group" once a year that may include broader representation from medical schools and student organizations. Many past members of this group report, however, that their collective concerns have not been seriously addressed.

The NBME's plan to implement the USMLE Step II CS has undergone massive changes over the past 5 years. For example, a few years ago, the NBME considered administering the exam in at least 45 medical schools rather than at independent testing centers, and there were no known plans to require the exam for medical licensure. According to the medical student liaisons to the NBME, the NBME disclosed in September 2001 that they were cutting back to 5 test sites and charging $1000 per student. In response, the students expressed concerns regarding equitable access to test sites and the financial burdens related to fees and travel. According to a liaison from the AMA student section, the NBME denied that the exam would be included in the USMLE Step II at a special meeting with the NBME in 2002.13 The student liaisons complained that the NBME's communication about the exam "has been inadequate."14 Frustrated, the AMA's Medical Student Section organized a grassroots campaign in 2002 encouraging medical students to take up these concerns with their state medical licensing boards. This campaign became one of the many driving forces shaping AMA policy regarding this exam.

Curricular enhancements for assessing clinical skills have received widespread support throughout medical education.15 However, some prominent organizations have expressed sincere concern regarding the implementation of the USMLE Step II CS and its use for the purpose of medical licensure. The American Association for Medical Colleges (AAMC) in 2002 called for a delay in the implementation of the exam until alternative funding sources could be identified to alleviate the financial burden for students. Last year, the AMA declared that the assessment of clinical skills is "best performed using a rigorous and consistent examination administered by the medical school" and "should not be used in evaluation for licensure of graduates of LCME- and AOA-accredited medical schools."16

The AMA's concerns are 2-fold: (1) the USMLE Step II CS may not allow remediation before residency for medical students who fail the test, and (2) the USMLE Step II CS should not be used for medical licensure because it does not accomplish its stated goal—adequately protecting the public from harm. The NBME has failed to provide evidence that their exam will improve clinical skills or patient safety beyond the level of current medical school-based assessments.17,18 Studies of the similar exam used for IMGs demonstrate that the test effectively assessed only the use of spoken English.19 For the sake of patient safety, the AMA argues that remediation for medical students with clinical skill deficiencies is best accomplished before those students take on additional patient care responsibility. Medical students who fail the exam may not have an opportunity for reexamination before starting residency. There is limited capacity available at the 5 testing centers that will service both US graduates and IMGs. Students who must repeat the exam may leave residency positions unfilled or take time away traveling to test centers, creating deficiencies in training and patient care.

While the NBME and FSMB leadership have heard the AMA's concerns, they have not taken meaningful measures to address them. The inability of the AMA, AAMC, and medical student groups to influence medical licensing reform has raised fears that professional organizations dealing with medical education have lost their ability to self-regulate as the pace of health care systemic change quickens. At the same time, the medical education system is under pressure to adjust rapidly to these challenges as the public demands greater accountability for health care quality. The primary goal of the formal evaluation of clinical skills throughout medical education is to augment clinical competence. Implementing a national exam for medical licensure intended to assess clinical skills using standardized patients may address the surface issues of communication skills and establishing patient rapport. Only reinforcing professional behaviors and ethical decision-making throughout formal medical education, however, will realistically reduce the number of physicians cited for licensure infractions.

References

  1. United States Medical Licensing Examination Web site. Clinical skills exam: FAQs. Accessed October 21, 2003.

  2. Vincent C, Young M, Phillips A. Why do people sue doctors? A study of patients and relatives taking legal action [comment]. Lancet. 1994;343(8913):1609-1613.
  3. Ambady N, Laplante D, Nguyen T, Rosenthal R, Chaumeton N, Levinson W. Surgeons' tone of voice: a clue to malpractice history. Surgery. 2002;132(1):5-9.
  4. Moore PJ, Adler NE, Robertson PA. Medical malpractice: the effect of doctor-patient relations on medical patient perceptions and malpractice intentions. West J Med. 2000;173(4):244-250.
  5. Stewart M, Brown JB, Boon H, et al. Evidence on patient-doctor communication. Cancer Prev Control. 1999;3(1):25-30.
  6. Waldman JD, Spector RA. Malpractice claims analysis yields widely applicable principles. Pediatr Cardiol. 2003;24(2):109-117.
  7. Kohn LT, Corrigan JM, Donaldson MS, eds; Committee on Quality of Health Care in America, Institute of Medicine. To Err is Human: Building a Safer Health System. Washington DC: National Academies Press; 2000.

  8. Kohn LT, Corrigan JM, Donaldson MS, Editors; Committee on Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington DC: National Academies Press. 2001.

  9. Reid WH. Recognizing and dealing with impaired clinicians, Part 1: Recognition and reporting. J Med Prac Manage. 2001;17(2):97-99.
  10. Scotti, Michael. AMA Senior Vice President for Professional Standards, 2003. Personal communication.

  11. Boulet J, McKinley D, Whelan GP, Van Zanten M, Hambleton RK. Clinical skills deficiencies among first year residents: utility of the ECFMG clinical skills assessment. Acad Med. 2002;77(10 Suppl):S33-S35.
  12. Stillman PL, Regan MB, Swanson DB, et al. An assessment of the clinical skills of fourth-year students at four New England medical schools. Acad Med. 1990;65(5):320-326.
  13. McCall, Chad. AMA Medical Student Section Liaison, NBME Medical Student Liaison Committee, March, 2002. Personal communication.

  14. National Board of Medical Examiners. Special Medical Student Liaison Meeting Minutes. Philadelphia, NBME Headquarters NBME Special Medical Student Liaison Meeting. 2001-2002.

  15. Liaison Committee on Medical Education. Function & Structure of a Medical School, 1990 accreditation standards. current.

  16. American Medical Association. H-275.956: Demonstration of Clinical Competence. (CME Rep. E, A-90; Reaffirmed: CME Rep. 5, A-99; Modified: Sub. Res. 821, I-02).

  17. Scoles, Peter, NBME Senior Vice President for Assessment. Communication to the AMA Medical Student section meeting. June, 2001.

  18. Melnick, Donald, NBME President. Communication to the AMA Council on Medical Education, December, 2001. Communication at meeting.

  19. Boulet JR, van Zanten M, McKinley DW, Gary NE. Evaluating the spoken English proficiency of graduates of foreign medical schools. Med Educ. 2001;35(8):767-773.

Citation

Virtual Mentor. 2003;5(12):572-576.

DOI

10.1001/virtualmentor.2003.5.12.pfor1-0312.

The viewpoints expressed in this article are those of the author(s) and do not necessarily reflect the views and policies of the AMA.