Three Things Schools Should Do to Make Advancement Assessment Just

Marjorie  Westervelt; Darius  Billingsley; Maya  London; Tonya  Fancher

doi:10.1001/amajethics.2021.937.

Medical Education

Dec 2021

Peer-Reviewed

Three Things Schools Should Do to Make Advancement Assessment Just

Marjorie Westervelt, PhD, MPH, Darius Billingsley, MD, Maya London, and Tonya Fancher, MD, MPH

AMA J Ethics. 2021;23(12):E937-945. doi: 10.1001/amajethics.2021.937.

Abstract

This article considers how student advancement assessment in American medical schools undermines equity. Although much attention is paid to admissions processes’ capacity to diversify the physician workforce, students’ advancement has been neglected as the next key step along their journeys toward graduation and residency training. This article canvasses common ways advancement undermines equity and suggests 3 areas of focus. In particular, it suggests that retention, student progression, and career advancement milestones are at least as important as admissions-based efforts to promote justice in medical education opportunity.

Introduction

In Regents of the University of California v Bakke (1978), the US Supreme Court acknowledged the educational benefits of diversity, writing that while racial quotas were unconstitutional, race-conscious admissions policies were legal if race was one of many factors considered.¹ Over 40 years later, we—affiliates of the medical school whose rejection of Bakke’s application spurred the case—revisit the concept of diversifying the physician workforce and ensuring that medical education systems’ support of diverse learners does not end with recruitment but continues throughout the educational continuum.²

Defining and Measuring Diversity

Medical schools’ social missions are measured by 3 indicators: the percentage of graduates practicing primary care, the percentage of graduates practicing in health professional shortage areas, and the percentage of graduates from backgrounds underrepresented in medicine (URiM).³ The historically Black colleges and universities dominate the social mission rankings by educating the vast majority of Black physicians in the United States and delivering a curriculum that inspires graduates to practice in locations and specialties with physician shortages.³ The importance of this work is magnified by recent evidence of improved health outcomes when Black patients are cared for by Black physicians.^4,5 There is a need for other schools to adopt similar strategies to promote diversity more broadly.

In 2003, the Association of American Medical Colleges (AAMC) broadened its definition of URiM to include any “racial and ethnic populations that are underrepresented in the medical profession relative to their numbers in the general population” to permit schools to define underrepresentation based on their region⁶ and to include nonracial and ethnic identities, such as sexual orientation, disability,⁷ rural origin,⁸ growing up in a low-income household, and first generation to attend college.⁹ Prior to 2003, underrepresented minority was the term the AAMC used to refer to “Blacks, Mexican-Americans, Native Americans (that is, American Indians, Alaska Natives, and Native Hawaiians), and mainland Puerto Ricans.”⁶ The shift from “underrepresented minority” to URiM helped medical schools to think more broadly and more regionally about workforce diversity.

How progress in diversity is measured is shaped by the mission of an institution and its ideal student population. In interviews with medical school admissions officers across the United States, Ko et al (unpublished data, 2019-2020) found that they take advantage of the local approach by creating their own definitions of diversity with easy-to-meet thresholds. For example, institutions can use the demographics of their surrounding community or state as diversity benchmarks. Thus, in states with less racial and ethnic diversity, the target number of URiM matriculants will be lower. However, if national—or even global—population data were used, the benchmark for diversity at any given institution would be higher.

Diversity benchmarks can be used to promote not only parity^10,11,12,13 or equality, whereby everyone receives the same thing regardless of their background, but also a culture of equity, which ensures that all students receive what they need to be successful and that all aspects of medical education are just. A culture of equity, however, cannot be limited to recruitment, admissions, and selection but must encompass curriculum, assessment, and career advancement milestones. Much of the focus in diversifying the physician workforce is on pathway programs to expand the applicant pool^14,15,16 and admissions processes such as multiple mini-interviews,¹⁷ trainings to mitigate bias,^18,19 and holistic review²⁰ based on applicants’ experiences, attributes, and metrics. Less research focuses on how to ensure a supportive and equitable learning environment for learners once they matriculate. We contend that progress in equity must be measured in terms of not only recruitment to medical school but also success in medical school and beyond.

Equity in Assessment

Although holistic admissions practices have greatly increased the number of students from minoritized groups attending medical school,²¹ the medical education system is lagging in developing and implementing strategies that ensure student success. This lag is not due to lack of motivation or intention; in fact, medical schools across the country have invested significant resources in providing academic support to students from minoritized groups who tend to matriculate with lower metrics (eg, grade-point average, Medical College Admission Test^®[MCAT] scores).²² However, it may be unreasonable to expect the same performance from students with diverse educational opportunities and experiences. Moving away from traditional performance expectations (which were set in place by historically dominant groups) and redefining success is the only way to achieve equity in assessment in medical education.

How progress in diversity is measured is shaped by the mission of an institution and its ideal student population.

Lucey et al classified equity in medical assessment as a “wicked problem,” that is, a problem that is immeasurably complex and nearly impossible to solve.²³ The authors identified 3 requirements for equity in assessment: intrinsic equity (unbiased assessments), contextual equity (a fair learning environment), and instrumental equity (the use of assessment data in advancement and selection).²³ Layering these equity requirements on a competency-based medical education (CBME) framework provides an opportunity to create a system for equitable advancement and progression. Per the first requirement, assessments in a medical school employing a CBME framework would use criterion-based measures of performance, which compare performance to a predetermined standard or performance level and provide students as much time as necessary to achieve them. The decision of the United States Medical Licensing Examination^® (USMLE) to move from a numeric to a pass/fail scoring system for Step 1, effective no earlier than January 1, 2022, is a timely example of aligning assessment with the principles of CBME.²⁴ This change might reduce the USMLE’s negative effects (eg, isolation, anxiety, and the misuse of Step 1 scores for residency selection or measurement of competence) on URiM students.²⁵

Standardized assessments (eg, USMLE Step 1 and Step 2 Clinical Knowledge) are one, albeit a predominant, traditional metric of success in undergraduate medical education (UME). Other “measures” of success include clerkship grading and clinical skills assessment—both of which can be extremely subjective and biased—and matching into a residency program. In a multi-institutional study of over 600 medical students, Bullock et al found that only 44% of medical students believed clerkship grading to be fair.²⁶ Taking a different approach, Teherani et al conducted semi-structured interviews with 20 senior medical students and residents to identify what they perceived as equitable assessment practices.²⁷ The analysis identified a number of possible improvements related to clinical assessment, including, but not limited to, shifting the focus from grades to patient care and removing peer comparisons. These findings once again reinforce the benefits of moving to a CBME model: focusing on the ultimate goal of the educational experience (ie, safe and effective patient care) and using criterion-based rather than normative standards. In response to this feedback, some schools decided to move to a pass/fail grading system for clerkships. However, as of the 2019-2020 academic year, only 11 of 153 schools had adopted this strategy.²⁸ A major challenge to widespread adoption of CBME is residency selection: most residency programs rely on clerkship grades to identify medical students worthy of consideration for their specialty. Without clerkship grades and USMLE Step 1 scores, residency programs would need to find a more holistic way to review applicants, which might improve the representation of URiM residents (and eventually faculty) across all specialties.^29,30,31

Standardized exams and residency selection are part of the larger medical education system that must be examined in order to achieve equity. Some students from URiM groups enter medical school having had less access to academic preparation and having underperformed on standardized tests due to the tests’ inherent biases.^32,33,34 This inequity can be traced throughout the K-12 and undergraduate education systems. Expecting students to make up 16 years of disadvantages in 2 years (or less in some medical schools) while learning and retaining all the new knowledge presented to them in the preclerkship curriculum is unrealistic. However, students who may (understandably) need more time face extreme scrutiny by student progress committees through repeated reviews of their academic progress and the need to justify delays, and viewing this extra time as a delay or falling behind can have a negative impact on their well-being.³⁵ Many URiM students also have fewer financial resources upon matriculation and leave medical school with more debt than their peers from higher socioeconomic backgrounds.^36,37 A more equitable CBME system would have a “flat rate” for the MD degree, allowing students as much time and resources as necessary to move through the curriculum. In this system, success would be defined as every student finishing medical school, regardless of the amount of time it takes.

Where Can We Go?

As gatekeepers to the profession, US medical schools should embrace their role in creating an equitable medical education system and in driving the representiveness and diversity of the workforce^{30,38,39,40,41,42} that will address health needs around the globe, following the examples below.

Recruitment with retention. While several US medical schools (as well as graduate medical education [GME] programs^29,43) have implemented holistic review and multiple mini-interviews to recruit a diverse student body,^44,45,46,47 some have also added GME to UME admissions, reenvisioning admission to medical school as admission to UME and to GME. For example, Oregon Health & Science University (OHSU) and the University of California, Davis (UC Davis), with grant support from an American Medical Association Accelerating Change in Medical Education initiative, established a collaborative known as COMPADRE (California Oregon Medical Partnership to Address Disparities in Rural Education and Health). OHSU and UC Davis, along with regional residency programs, co-recruit and train the physicians needed in the rural, tribal, and urban communities residing between Portland, Oregon, and Sacramento, California.⁴⁸ Other organizations have approached UME/GME joint recruitment through a time-variable approach. For example, the Education in Pediatrics Across the Continuum Project bases advancement on the achievement of competency rather than time-based milestones across the UME-GME continuum.⁴⁹

Retention and advancement. The 2021 Coalition for Physician Accountability’s 42 recommendations to improve the UME-to-GME transition include a call to action for UME and GME programs to eliminate systemic biases in their grading and awards structures.⁵⁰ The profession’s collective overreliance on metrics to assess student performance has been dispelled by the Morehouse School of Medicine. The Step 1 scores of students graduating between 2009 and 2014, who received interventions designed to facilitate success, exceeded those expected based on their MCAT scores.²² The school creates the right milieu for learning as well as mentoring opportunities, aligns the structure and content of its curriculum to its mission, and uses a robust system to monitor student performance and retention. Similarly, the University of Michigan School of Medicine is leading the nation in providing an inclusive environment for students with a physical disability or functional limitation⁷ and in intentionally aiming to reduce barriers for learners to promote equity in access and education. Although not directly supporting retention, the AAMC’s application to medical school offers prospective students the option of specifying gender identity and preferred pronouns,⁵¹ and the Accreditation Council for Graduate Medical Education awards⁵² includes awards for diversity and inclusion. These are 2 major steps in recognizing the diverse identities of medical learners and in promoting inclusion.

Career advancement. Exemplars of equity in access to medical specialty careers are harder to identify. The Indiana University School of Medicine publishes a diversity dashboard fact sheet— including data for UME students, staff, faculty, GME trainees, and the state of Indiana—which is a step toward accountability (though the current dashboard does not include data by specialty).⁵³ Both Ohio University Heritage College of Osteopathic Medicine and the University of North Carolina (UNC) offer programs that include post-GME retention at the point of entry to medical school. The Heritage College’s Transformative Care Continuum is an accelerated UME-to-GME program that includes a contract with the Cleveland Clinic upon residency completion.⁵⁴ The UNC Fully Integrated Readiness for Service Training program is a UME-to-GME program that includes 3 years of post-GME service in rural and underserved North Carolina.⁵⁵

Conclusion

We propose that defining diversification goals at individual institutions demands that those institutions honor regional needs to provide the best care,^4,5 advance health equity,⁵⁶ and optimize the educational benefit for all students.^1,57 We encourage schools to embrace recruiting a health care workforce that is diverse with respect to race and ethnicity but also to consider identities such as disability, sexual orientation, socioeconomic status, and first generation to attend college, and the intersection of these identities. Once students have been recruited, schools must support them throughout their education, remove barriers to equitable advancement, encourage them to explore all specialties, and continue to support them as they transition to careers.

AMA Journal of Ethics^®

Illuminating the Art of Medicine

Three Things Schools Should Do to Make Advancement Assessment Just

Abstract

Introduction

Defining and Measuring Diversity

Equity in Assessment

Where Can We Go?

Conclusion

Read More

References

Also in this Issue

Dec 2021

Whose Responsibility Is It to Address Bullying in Health Care?

How Should Lived Experience of Racism Count in Medical School Admissions?

What Is the Role of Accreditation in Achieving Medical School Diversity?

Does the AAMC’s Definition of “Underrepresented in Medicine” Promote Justice and Inclusivity?