AMA Journal of Ethics^®

Under the banners of “holistic review” and “mission-centered admissions,” medical schools across the nation are increasingly considering a broader array of student characteristics in constructing a student body to meet the missions of medical education and the health care needs of the public. Estimates from the American Association of American Medical Colleges (AAMC) and the American Medical Association suggest a growing shortage of physicians in the U.S., especially in primary care and general surgery [1, 2]. These trends in medical education and the health care system call into question traditional medical school admission policies, which tended to recruit an archetypal student on the basis of narrowly defined academic characteristics (i.e., MCAT performance and grade point averages), which may not be especially relevant in meeting the public’s need for primary care physicians interested in promoting patient health and well-being.

Changes in the aims of medical education and the health care system are outpacing the innovations needed to support them. The medical education establishment can make fairly accurate predictions of applicants’ future academic performance, using these innovations to select students who are likely to achieve academic success as indicated by course grades, completion of medical school, and passing the United States Medical Licensing Exams. The medical education system, however, lacks widely accepted instrumentation to predict such things as specialty choice, propensity to provide care to the medically underserved, future clinical competence with a diverse array of patients and contexts, and a number of other core characteristics needed to meet the missions of medical education and the health care needs of communities. While scientifically validated tools are emerging, it has not been clearly established to what degree medical education should apply these innovations to construct incoming cohorts of students and to socialize and train them thereafter.

Holistic review is increasingly being recognized as a philosophy that can be employed to account for not only students’ positive qualities but also society’s needs [3]. The AAMC defined holistic review as a “flexible, highly-individualized process by which balanced consideration is given to the multiple ways in which applicants may prepare for and demonstrate suitability as medical students and future physicians” [4]. Holistic review affords medical schools the ability to better meet such missions as admitting a culturally and economically diverse student body or graduating more students who enter primary care or other specialties in shortage areas. Groups such as the Innovation Lab with MR5 at the AAMC, which was responsible for the changes to the fifth edition of the MCAT, support the concepts underlying holistic review, as indicated by their recommendations to seek out students who possess “dependability, respect, altruism, [and] empathy” and to develop validated methods to assess applicants’ personal characteristics as part of a centralized application process [5].

Typical medical school admissions efforts ask only informally about students’ specialty intentions and interests, and guidance efforts to assess specialty interest levels begin only after students enter the clinical curriculum. An attempt to get valid information about medical students’ specialty interests to inform the admissions process, the Primary Care Orientation Scale (PCOS) was based on previous research [6] and designed as a comprehensive, multivariate, multidimensional, open-source approach to predicting medical students’ specialty choices. The PCOS includes assessments of inventoried medical interests, personality characteristics, demographics, academic performance, and vocational identity, all of which may be correlated with specialty choice. It is meant to provide an indication of students’ predisposition to choose primary care fields at an earlier-than-usual point in their education [7]. The PCOS accounts for both expressed interests—the specialty a student states that he or she is likely to go into—and inventoried interests—scales of clinical activity interests that may be predictive of specialty interest or aptitude [6].

Studying Predictive Methods

Because scales that account for expressed interests tend to more closely mirror a student’s ultimate specialty choice than do inventoried interests [8], we investigated the association between the expressed and inventoried interests of students who had successfully navigated the admissions process and were pursuing medical degrees.

We recruited first-, second, and third- year students at a medical school in the Midwest via school e-mail to take part in the study. Each student had a private link to the survey, which was available online for a 2-week period. There were a total of 211 questions on varied topics such as biomedical and biopsychosocial interests, medical specialty identity, work values, financial aspects of life and work, personality characteristics, general abilities, and work and family role expectations. Examples of questions asked on the PCOS include:

1. Indicate how much you think you would like or dislike the following:

• Using a highly detailed understanding of the body.
• Providing primary care services.
• Establishing close relationships with patients.

2. When I become a physician, it will be important for me to:

• Have a predictable and stable work life.
• Be recognized by others as being in a prestigious field.
• Make a positive impact on the lives of those around me.

3. How much has your debt influenced your career decisions?

4. To what extent do you agree with the following statements?

• Becoming a physician in my chosen specialty will allow me to become the person I dream to be.
• Thinking about choosing a medical specialty makes me feel uneasy.

After students took the survey, we added information obtained from the medical school database: student demographics such as age, sex, and ethnicity; academic data such as MCAT scores, USMLE Step 1 and 2 scores when applicable, grades (on an honors/pass/fail system); and any years of school retaken or leaves of absence.

Initial analyses of the data suggest that students’ inventoried interests (survey items about medical activity interests) are stronger predictors of their expressed interests (anticipated specialty choice) than any of the other variables from the survey. However, the capacity of inventoried interests to predict expressed interests varies with certain student characteristics. Specifically, inventoried medical specialty interests were more predictive of expressed interests for students from medical families (e.g., parents who are physicians) and those who enrolled in accelerated BS/MD medical school programs than for students from nonmedical families. This suggests that the interests of some students from particular backgrounds and who choose certain kinds of medical training pathways may be more informed than those of their peers. This finding has implications for holistic admissions processes that bank on responses to such questions as “What medical specialty are you leaning toward?” by suggesting that the validity of students’ responses may be shaped by their resources and choices.

The methodologies behind the PCOS have some limitations. They rely heavily upon students’ perceived interests and their willingness to report their interests faithfully. While the early results are promising, we have not tested the validity and accuracy of the tool in indicating applicants’ interests before admissions decisions occur, and testing on applicants is needed. Also, while the established association between inventoried and expressed interests is promising, the current research on the PCOS does not establish a relationship between interests and eventual residency choices, which will need to be investigated to ensure that the PCOS is predictive of behavioral outcomes.

Ethical Analysis

At this point it is difficult to assess the ethical implications of employing instruments like the PCOS in medical school recruitment and admissions. We believe that it is fair to assume that schools vary in their willingness to share their intentions with prospective students. While no known published research has examined prospective students’ perception of such instruments, anecdotal evidence of those who have applied to medical school in the last few years suggests that at least some fraction of this group has been questioning the motivations that underlie the inventories and tests that they are being asked to undergo for the sake of entering medical school. Our experience suggests that some applicants have expressed concern about the rigor and validity of the methodology through which the scales are used to inform admissions decisions.

Unfortunately, revealing more to prospective students would make it possible for them to prepare for the tests, in essence stripping the assessments of their “blinding” and, therefore, a large measure of their utility. Given the competitive aspects of medical school admissions, some students may necessarily feel inhibited to ask questions about the validity of the measures they are required to complete. However, because many of these scales are novel and students have little familiarity with the research supporting their use, they should be encouraged by admitting committees and teams to ask such questions and express concerns if they have them.

Our conclusion is that, when a school is employing mission-centric criteria to choose among students with satisfactory academic records, it may be ethical to take into account attributes such as specialty interest, emotional intelligence, communication skills, and diversity. We believe that this would be particularly true when the instrumentation assessing mission-oriented characteristics has strong evidentiary support from studies of applicants under the same conditions. We suspect, however, that many nonacademic mission-oriented instruments currently in use may lack such evidence, and we encourage medical admissions committees to seek the counsel of measurement experts before administering or continuing to administer such measures.