Case and Commentary
Jan 2024
Peer-Reviewed

Why Competency Frameworks Are Insufficiently Nuanced for Health Equity Teaching and Assessment

Zareen Zaidi, MD, PhD, Daniele Ölveczky, MD, MS, Nicole A. Perez, PhD, Paolo C. Martin, PhD, Andres Fernandez, MD, MSEd, Philicia Duncan, MD, and Hannah L. Anderson, MBA
AMA J Ethics. 2024;26(1):E12-20. doi: 10.1001/amajethics.2024.12.

Abstract

This article suggests how competency-based medical education should robustly integrate health equity by focusing on physicians’ responsibilities to (1) know why and how underlying structural mechanisms contribute to health equity and then (2) take action to achieve health equity in their practice. This article first canvasses currently available frameworks for helping trainees cultivate these 2 specific skills of discernment and action. This article then offers strategies for teaching and assessing these skills in specific learning activities.

Case

Dr A is a general pediatrician working alongside Dr B, a pediatric intern at a primary care clinic. Today is Dr A’s first day working with this intern, but Dr A has already been impressed with Dr B’s skills and attention to detail. Dr B says: “David is a 6-year-old boy here for a well-child visit. Dad is here today; he reports issues at school, including suspensions. His school reports suspensions are related to behavioral concerns: aggression, inattention, tearfulness. Dad reports concerns about bullying and says managing his boy’s behavior at school has been difficult.” Although Dr A has confidence in the intern’s clinical skills, Dr A wonders if Dr B can incorporate in David’s care the social determinants of health that contribute to his presentation today. Dr A has had experience with families like David’s and knows that children of color like David often face interpersonal, structural, and internalized racism that affects their behavioral health.1,2 Moreover, Dr A is aware of the well-documented racial disparities in behavioral management and school suspensions.3,4,5

How might Dr A appropriately assess this intern’s developing awareness and skills in asking, listening, and addressing these potential racial disparities? Which, if any, existing competency literature could help Dr A teach this intern effectively about health equity and how to integrate it into David’s care? While Dr A is aware of numerous ways to assess general clinical competencies, such as examinations and direct observation, Dr A is not comfortable in assessing health equity skills.

Commentary

The origins of competency-based medical education (CBME) can be traced to a school of psychology called behaviorism,6 which focused on documenting observable outcomes and ensuring that these outcomes served as the determining factor in decisions about curricula.7 CBME works to create content, experiences, and assessments based on observable outcomes to develop the necessary knowledge, skills, attitudes, and behaviors or actions that support a particular competency.8 The first competency framework, the CanMEDS Physician Competency Framework, which was established in 1996, outlined 7 competencies intended to improve patient care.9 Other efforts to enhance competency training and assessment soon followed, including the Accreditation Council for Graduate Medical Education (ACGME) Outcomes Project in 1998 that led to the ACGME’s 6 core competencies (patient care, medical knowledge, practice-based learning and improvement, systems-based practice, professionalism, and interpersonal skills),10 and the Association of American Medical Colleges (AAMC) 13 Core Entrustable Professional Activities (EPAs) for Entering Residency in 2014.11

Although CBME is a dominant movement in medical education, CBME frameworks did not originally define outcomes related to health equity or assess content areas related to structural determinants of health.12 Yet there is agreement in the academy and professional organizations that individual physicians should have knowledge of health equity and do have a professional responsibility to realize it in their practice in order to affect patient outcomes.13,14 Beyond the few efforts made to amend competency-based frameworks,12 health equity is typically an afterthought in curricula that were not designed to teach or assess the critical knowledge that is needed to understand and advance health equity.12

In keeping with CBME frameworks’ artificial distinctions between knowledge and action, the current approach is to teach about inequities rather than to expose why and how underlying structural mechanisms (eg, racism, sexism, ableism, violence, differential education and employment opportunities, health care access) contribute to such health inequities.15,16,17 By contrast, other health professions have built foundational competencies related to health equity knowledge and action into their accreditation standards for degree-granting programs.18 In social work, for example, trainees are expected to advance social, racial, economic, and environmental justice and human rights, as well as to engage in antiracist practice and diversity, equity, and inclusion efforts by developing empathy and awareness of White privilege and taking responsibility for the structural mechanisms underlying disparities.18,19,20 Given that both social work and medicine are professions with foundations in caring, why are their competencies so starkly different? Medical education fails to foreground health equity as an expected part of professional knowledge and action. Thus, a critical shift is needed in the way we think about health equity in the context of CBME.

Defining a Health Equity Skill Set

There is evidence that CBME improves procedural skills,21 but there is ongoing debate regarding its suitability to assess the “development of values, insight, and judgment.”6 An emphasis on proficiency that is based on measurement of observable behaviors reduces the “holistic expertise” of physician training to “a series of discrete tasks” and ignores the connections that make these tasks “a purposeful whole.”22

Are competency frameworks—which are designed to reduce complexity—sufficient to capture such a nuanced concept as health equity? 

Consider a widely used competency framework in general pediatrics: the pediatric milestones. The “systems-based practice” subcompetency, “population and community health,” level 1 states that trainees should demonstrate “awareness of population and community health needs and disparities” and, by level 4, adapt “practice to provide for the needs of and reduce health disparities of a specific population.”23 In this competency framework, trainees are currently only expected to know about general health disparities, not the underlying structural mechanisms (eg, racism, sexism, ableism) that create those health disparities. In the absence of an understanding of power dynamics, how can interns realistically “adapt” their practice to patients’ needs, or how can educators teach trainees to—and assess their ability to—address and mitigate health inequities?

Health equity is a complex concept with multiple layers,13 including various types of health outcomes, which intersect various medical specialties, as evidenced by the pediatrician in the vignette seeking to address inequities in behavioral health outcomes related to racism. This complexity presents a challenge for CBME. Are competency frameworks—which are designed to reduce complexity—sufficient to capture such a nuanced concept as health equity?

Reimagining Assessment

In line with others who have criticized the foundations of CBME,6,22 we contend that building solid health equity education will require going beyond the atomization of skills and the artificial separation between knowledge and action. In contrast to others who have argued that CBME can be retrofitted for health equity,12 we argue that because the CBME movement was never built to address health equity, its foundations are critically lacking for the purpose. The question then becomes: Can and should CBME be used to assess a trainee’s ability to address and mitigate inequities? If CBME does not serve this purpose, then what does? If the traditional assessment tools associated with competency frameworks are not enough, then what is? Below, we draw on scholarship from other professions, including education, social work, and nursing, to provide some building blocks that we believe will aid in reimagining and rebuilding new foundations for medical education for health equity.

Valuing advocacy and community-based work as central to the development of professionals. Assessment systems comprise the multiple assessment tools and methods used to measure and support learning and produce comprehensive data about learners’ development.23,24 Assessment systems are a direct communication of an institution’s values to learners and other stakeholders, including patients and community members. Therefore, assessments should be based on values such as learners’ growth, rather than on ranking and sorting learners, and reflect the needs of the community.25 The most well-known assessment tools for CBME include multiple choice questions, direct observation, performance data, multi-source feedback, simulation exercises, and reflective writing. What these assessments often fail to value are the settings and skills necessary for teaching and assessing health equity. While a simulation exercise could be used to assess learners’ advocacy skills, it would not give insight into their understanding of how and why structural racism affects patients or, more importantly, the real-time actions that they can and should undertake. Reflection has also fallen into the trap of being viewed as a measurable construct. Specifically, reflective writing exercises run the risk of the trainee generating the desired response to the prompt, which may not be an honest reflection of their true feelings on the matter.26 A more productive way forward is to design learning opportunities that focus on allowing learners to understand how health disparities have come into existence by actively providing them with platforms to engage in community-based and advocacy work. Proximity to health disparities through longitudinal community engagement can help learners understand their role as a physician in their community.15

Understanding how structural racism plays a role in standardized testing. There is mounting critique of how standardized tests serve as “institutionalized mechanisms of racial exclusion.”27 The fact that underrepresented learners get lower scores on standardized exams27 is a product of a multitude of historical circumstances—such as redlining, suboptimal schools, and intentionally disinvested neighborhoods—which lead to lower test scores. Admissions committees may in turn place too much emphasis on these scores. We argue that allowing learners to pave their own path for community engagement without mandatory quantitative assessments could result in a more authentic learning experience. In contrast to conventional medical education wisdom, we contend that everything that needs to be learned does not have to be measured—at least, not in a standardized or normative way—to avoid a potential quantitative fallacy.28 Deemphasizing quantification gives room for alternative approaches to building assessment systems, such as assessments for inclusion29 and justice-orientation.30,31  These approaches recognize the roles of structural racism in education—notably, that historical educational practices such as standardized testing were designed by and for individuals from dominant race, class, gender, and ability groups.30,31 Furthermore, these approaches recognize that underrepresented learners often enter educational spaces both with prior experience of inequity and with unique strengths that both go unacknowledged.

Introducing creativity, flexibility, and individualization into assessments. Learners are not homogenous, and it is counterproductive to teach and assess all learners uniformly. As assessments for inclusion and justice orientation recommend, allowing learners to design their learning agenda and assessments as stakeholders would likely be more insightful than a top-down assessment rubric related to their understanding of health inequities. Other professions routinely use assessment for inclusion practices, such as “un-grading,” which intentionally focuses more on learning and less on grades and performance.32 Other examples include faculty involving trainees in designing health equity assessments, such as portfolios of work; partnering with them to design the assessment system itself; allowing multiple ways to demonstrate learning with minimal grading; and inviting trainees’ feedback on assessment processes.33,34

Ensuring assessments are culturally relevant and valuing awareness of the historical and structural basis of current inequities. One practice in justice-oriented assessment—culturally relevant education and assessment—allows students to draw connections between their learning and their lived experiences.31 One example might include implementing capstone longitudinal community projects (identified and undertaken by trainees), with a group presentation and sharing of lived experiences throughout the project, which could be a powerful tool for learning.35,36,37 Designing culturally relevant assessment tools and systems would require using systematic assessment design approaches that engage patients and community members alongside learners in designing assessment tools. It would also require using equity-centered program evaluation models to evaluate the design’s influence on communities.

Returning to the case scenario above, how do the 4 proposed health equity-assessment building blocks compare with existing CBME assessment tools? The table below demonstrates how these health equity-assessment blocks, applied to David’s case, allow for a deeper engagement with the causes of health inequities and prompt learners to develop community-based solutions.

table1-cscm2-2401

Conclusion

The dominant discourses around CBME value knowledge production and assessment according to their conformity with a “zero-point epistemology” or, in simplified terms, their “objectivity.” Yet putatively objective CBME assessments have been riddled with concerns about bias.38,39 Recognizing the limitations of CBME and its assessment tools, we have provided suggestions for moving beyond health equity as an “add-on,” utilizing practices from other professions—and envisioning other practices—that center equity as a foundational part of professional knowledge and action.

References

  1. Meek SE, Gilliam WS. Expulsion and suspension in early education as matters of social justice and health equity. NAM Perspect. October 31, 2016. Accessed September 28, 2023. https://nam.edu/expulsion-and-suspension-in-early-education-as-matters-of-social-justice-and-health-equity/

  2. Spencer N, Raman S, O’Hare B, Tamburlini G. Addressing inequities in child health and development: towards social justice. BMJ Paediatr Open. 2019;3(1):e000503.

  3. Sabol TJ, Kessler CL, Rogers LO, et al. A window into racial and socioeconomic status disparities in preschool disciplinary action using developmental methodology. Ann N Y Acad Sci. 2022;1508(1):123-136.
  4. Okonofua JA, Walton GM, Eberhardt JL. A vicious cycle: a social-psychological account of extreme racial disparities in school discipline. Perspect Psychol Sci. 2016;11(3):381-398.
  5. Amemiya J, Mortenson E, Wang MT. Minor infractions are not minor: school infractions for minor misconduct may increase adolescents’ defiant behavior and contribute to racial disparities in school discipline. Am Psychol. 2020;75(1):23-36.
  6. Morcke AM, Dornan T, Eika B. Outcome (competency) based education: an exploration of its origins, theoretical basis, and empirical evidence. Adv Health Sci Educ Theory Pract. 2013;18(4):851-863.
  7. Harden RM, Crosby JR, Davis MH. AMEE guide no. 14: outcome-based education: part 1—an introduction to outcome-based education. Med Teach. 1999;21(1):7-14.
  8. Competency-based medical education (CBME). Association of American Medical Colleges. Accessed February 17, 2023. https://www.aamc.org/about-us/mission-areas/medical-education/cbme

  9. About CanMEDS. Royal College of Physicians and Surgeons of Canada. Accessed February 17, 2023. https://www.royalcollege.ca/ca/en/canmeds/about-canmeds.html

  10. Swing SR. The ACGME Outcome Project: retrospective and prospective. Med Teach. 2007;29(7):648-654.
  11. The Core Entrustable Professional Activities (EPAs) for entering residency. Association of American Medical Colleges. Accessed February 17, 2023. https://www.aamc.org/about-us/mission-areas/medical-education/cbme/core-epas

  12. Unaka NI, Winn A, Spinks-Franklin A, et al. An entrustable professional activity addressing racism and pediatric health inequities. Pediatrics. 2022;149(2):e2021054604.

  13. Braveman P. What are health disparities and health equity? We need to be clear. Public Health Rep. 2014;129(suppl 2):5-8.
  14. Montoya-Williams D, Peña MM, Fuentes-Afflick E. In pursuit of health equity in pediatrics. J Pediatr X. 2020;5:100045.

  15. Sharma M, Pinto AD, Kumagai AK. Teaching the social determinants of health: a path to equity or a road to nowhere? Acad Med. 2018;93(1):25-30.

  16. Chokshi DA. Teaching about health disparities using a social determinants framework. J Gen Intern Med. 2010;25(suppl 2):182-185.
  17. Kuper A. When I say… equity. Med Educ. 2016;50(3):283-284.
  18. Commission on Accreditation; Commission on Educational Policy. 2022 Educational policy and accreditation standards for baccalaureate and master’s social work programs. Council on Social Work Education; 2022. Accessed February 17, 2023. https://www.cswe.org/getmedia/bb5d8afe-7680-42dc-a332-a6e6103f4998/2022-EPAS.pdf

  19. Olcoń K, Gilbert DJ, Pulliam RM. Teaching about racial and ethnic diversity in social work education: a systematic review. J Soc Work Educ. 2020;56(2):215-237.
  20. Owens-King AP, Anthony B, Venable VM, Jewell JR. Measuring undergraduate social work students’ knowledge and understanding of privilege and oppression. J Soc Work Educ Pract. 2020;5(3):55-75.
  21. Manthous CA. On the Outcome Project. Yale J Biol Med. 2014;87(2):213-220.
  22. Brooks MA. Medical education and the tyranny of competency. Perspect Biol Med. 2009;52(1):90-102.
  23. Accreditation Council for Graduate Medical Education. Pediatric milestones. Accreditation Council for Graduate Medical Education; 2015. Accessed August 29, 2023. https://www.acgme.org/globalassets/pdfs/milestones/pediatricsmilestones.pdf

  24. Norcini J. What’s next? Developing systems of assessment for educational settings. Acad Med. 2019;94(11)(suppl):S7-S8.
  25. Teherani A, Perez S, Muller-Juge V, Lupton K, Hauer KE. A narrative study of equity in clinical assessment through the antideficit lens. Acad Med. 2020;95(12)(suppl):S121-S130.
  26. de la Croix A, Veen M. The reflective zombie: problematizing the conceptual framework of reflection in medical education. Perspect Med Educ. 2018;7(6):394-400.
  27. Nguemeni Tiako MJ, Ray V, South EC. Medical schools as racialized organizations: how race-neutral structures sustain racial inequality in medical education—a narrative review. J Gen Intern Med. 2022;37(9):2259-2266.
  28. Carmody JB. On residency selection and the quantitative fallacy. J Grad Med Educ. 2019;11(4):420-421.
  29. Ajjawi R, Tai J, Boud D, Jorre de St Jorre T, eds. Assessment for Inclusion in Higher Education: Promoting Equity and Social Justice in Assessment. Routledge; 2022.

  30. Randall J, Slomp D, Poe M, Oliveri ME. Disrupting white supremacy in assessment: toward a justice-oriented, antiracist validity framework. Educ Assess. 2022;27(2):170-178.
  31. Randall J. “Color-neutral” is not a thing: redefining construct definition and representation through a justice-oriented critical antiracist lens. Educ Meas. 2021;40(4):82-90.
  32. Blum SD, ed. Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). West Virginia University Press; 2020.

  33. Ungrading: reimagining assessment of student learning. Columbia Center for Teaching and Learning. Accessed February 27, 2023. https://ctl.columbia.edu/faculty/sapp/ungrading/

  34. Odukoya EJ, Kelley T, Madden B, Olawuni F, Maduakolam E, Cianciolo AT. Extending “beyond diversity”: culturally responsive universal design principles for medical education. Teach Learn Med. 2021;33(2):109-115.
  35. Ladson-Billings G. Toward a theory of culturally relevant pedagogy. Am Educ Res J. 1995;32(3):465-491.
  36. Gay G. Culturally Responsive Teaching: Theory, Research, and Practice. 3rd ed. Teachers College Press; 2018.

  37. Paris D, Alim HS, eds. Culturally Sustaining Pedagogies: Teaching and Learning for Justice in a Changing World. Teachers College Press; 2017.

  38. Paton M, Naidu T, Wyatt TR, et al. Dismantling the master’s house: new ways of knowing for equity and social justice in health professions education. Adv Health Sci Educ Theory Pract. 2020;25(5):1107-1126.
  39. Teherani A, Hauer KE, Lucey C. Can change to clerkship assessment practices create a more equitable clerkship grading process? Acad Med. 2019;94(9):1262-1263.

Citation

AMA J Ethics. 2024;26(1):E12-20.

DOI

10.1001/amajethics.2024.12.

Conflict of Interest Disclosure

Authors disclosed no conflicts of interest.

Views expressed herein are those of the authors and do not necessarily express United States federal government policy. The people and events in this case are fictional. Resemblance to real events or to names of people, living or dead, is entirely coincidental. The viewpoints expressed in this article are those of the author(s) and do not necessarily reflect the views and policies of the AMA.