Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Working with entrustable professional activities in clinical education in undergraduate medical education: a scoping review

BMC Medical Education volume 21, Article number: 172 (2021) Cite this article

Abstract

Background

Entrustable professional activities (EPAs) are increasingly used in undergraduate medical education (UME). We conducted a scoping review to summarize the evidence for the use of EPAs in clinical rotations in UME.

Methods

We searched multiple databases for scoping reviews based on the PRISMA guidelines for articles reporting qualitative and quantitative research, as well as conceptual and curriculum development reports, on EPAs in UME clinical rotations.

Results

We identified 3309 records by searching through multiple databases. After the removal of duplicates, 1858 reports were screened. A total of 36 articles were used for data extraction. Of these, 47% reported on EPA and EPA-based curriculum development for clerkships, 50% reported on implementation strategies, and 53% reported on assessment methods and tools used in clerkships. Validity frameworks for developing EPAs in the context of clerkships were inconsistent. Several specialties reported feasible implementation strategies for EPA-based clerkship curricula, however, these required additional faculty time and resources. Limited exposure to clinical activities was identified as a barrier to relevant learning experiences. Educators used nationally defined, or specialty-specific EPAs, and a range of entrustability and supervision scales. We found only one study that used an empirical research approach for EPA assessment. One article reported on the earlier advancement of trainees from UME to graduate medical education based on summative entrustment decisions.

Conclusions

There is emerging evidence concerning how EPAs can be effectively introduced to clinical training in UME. Specialty-specific, nested EPAs with context-adapted, entrustment-supervision scales might be helpful in better leveraging their formative assessment potential.

Peer Review reports

Background

Entrustable professional activities (EPAs) have been internationally implemented in graduate medical education (GME) as units of clinical activities that can be entrusted to medical trainees in the clinical workplace [1]. Given that the clinical learning trajectory begins well before entering residency, medical educators have begun to explore how to apply EPAs into undergraduate medical education (UME) as well [2]. Ideally, the intensity of supervision and, inversely, the degree of independence, should change during and with each clinical rotation of medical students. EPAs, as competency-based learning goals, take a central place within competency-based curricula in medical education. However, the rapidly growing literature on EPAs in UME has not been synthesized from the perspective of clinical educators in UME.

Globally, clinical rotations represent a foundational element in UME [3]. However, the degree of students’ active participation in both the workplace and curricular design differ considerably across medical education systems [3]. Recently, longitudinally integrated clerkships have been recommended to provide educational continuity [4]. Currently, most medical schools still work with clinical rotations that are limited to a few weeks within given specialties [5]. From an assessment perspective, using summative assessments that are limited to four- to six-week clerkships is not ideal. However, short clinical rotations do provide important formative learning and assessment opportunities for medical students [6]. It is in this context that EPAs could be used to scaffold longitudinal and formative assessment systems, even if clerkship curricula are not longitudinally integrated [7]. Assessment data points collected during these clinical rotations could also inform summative entrustment decisions within overarching assessment programs [8].

Clinical educators represent a key stakeholder group when it comes to working with EPAs in the clinical workplace. In the review by Shorey et al. [9], the authors suggested that educators engaging with competency-based medical education should consider development, implementation, and the assessment of EPAs as discrete steps, and that stakeholder specific recommendations are needed. The review of Meyer et al. [2] addressed the question of assessing EPAs in UME and revealed the inconsistent use of terms and concepts related to EPAs. Furthermore, in general, quality criteria for good assessments have not been sufficiently considered for EPAs in UME.

In light of these previous reviews, the rapid growth of literature concerning EPAs, and emerging research on the pre-clerkship use of EPAs as learning goals [10, 11] we searched the literature for articles that provided evidence on how to work with EPAs in workplace-based learning contexts in UME. Specifically, we tried to address the evidence concerning how to develop, implement, and assess EPAs in different specialties and educational systems from a clinical educator’s perspective in UME and to contrast these findings with current best practice recommendations in GME. We also aimed to highlight important research gaps that should be addressed to advance the field of competency-based education with EPAs in UME clinical training.

Methods

We conducted a scoping review and followed the PRISMA-ScR Checklist to report our findings [12]. This approach was taken as the literature on EPAs in clerkships is still limited and emerging. Since the nomenclature for UME clinical rotations differs across educational systems, we included articles on clerkships, subinterships, and acting internships that are similar to the ‘practical year’ in Germany or the ‘elective year’ in Switzerland. Bootcamps were excluded as they typically do not include participation in daily clinical practice.

Data sources and search strategy

Our guiding review question was: “How have EPAs been used in clinical rotations in UME with regard to development, implementation, and assessment?” We reassessed the body of literature after conducting a search of the databases: PubMed, Cochrane Library, ERIC, Embase, PsycINFO, all Ovid journals, Scopus, Web of Science, and MedEdPORTAL. The search started December 1st 2018 and went until January 15th 2019, with a search update on August 6th 2019 using the search terms [“entrustable professional activity” AND “entrustable professional activities”] for each database for publications from 2005 (the introduction of EPAs) until 2019.

Screening and selection of articles

Inclusion and exclusion criteria are listed in Supplemental Table 1. To capture relevant EPA-based curricula in clinical workplaces in UME, we included articles related to clerkships, electives, subinternships, acting internships, and practical or transitional years. We imported all citations into EndNote X9 (Clarivate Analytics, Philadelphia, PA, USA). After the removal of duplicates, all items were screened for eligibility. First, titles and abstracts were considered, and duplicates removed (Fig. 1). We used the online software ‘Rayyan’ to select articles for full text reviews and resolve initial selection conflicts [13]. Two raters (SP and EL) independently screened all titles and abstracts in the review software. Conflicts were identified and resolved through discussion between raters. The final selection of articles was subjected to full text reviews. These were then used for data extraction and synthesis using a published data extraction form [1] to ensure comparability of our review. The final selection was cross-checked with the predefined inclusion and exclusion criteria.

Fig. 1
figure 1

Flow-chart of the scoping review process

Full size image

Data extraction and synthesis of results

National EPA-frameworks for UME were charted in a table and EPA content was compared (Table 2). We further applied codes for the main categories of EPA development, implementation, and assessment as described in the literature [1] (Supplemental Table 2). Codes were added iteratively. We focused on extracting data that were considered directly relevant for clinical rotation curriculum design based on the authors’ consensus (Supplemental Table 3). Since we conducted a scoping review (and due to the heterogeneous nature of the studies) we did not assess the quality of the studies.

Results

Search results

A total of 3309 articles were identified through database searches, and 1858 titles and abstracts were screened after the removal of duplicates. The screening resulted in 1677 articles being removed. There were 181 articles included in the full text assessment. A final selection of 36 articles was used in the data extraction. We found 20 newly published articles that had not been included in previous reviews on EPAs in the context of UME. The flow of the database search is shown in Fig. 1.

Summary of study characteristics

The included studies were summarized according to country of origin, type of article, specialty focus, and main focus in terms of EPA development, implementation, and assessment (Table 1). All of the articles originated from American and European countries, and the studies were predominantly carried out at single academic institutions. One study was conducted as a two-center study in the Netherlands and Hungary. We found an overall increasing trend of publication quantity per year. In terms of article type, the majority were categorized as expert consensus articles followed by educational case reports and cohort studies. Most articles did not focus on a specific specialty; however, those that did were mostly in the context of internal medicine, general surgery, and emergency medicine. The studies typically focused on one or two of the following aspects: development, implementation, or the assessment of EPAs in UME clinical rotations.

Table 1 Summary of article characteristics (n = 36) included in the scoping review on entrustable professional activities in clinical rotations in undergraduate medical education
Full size table

National EPA-frameworks for clinical curriculum design in UME

Three national organizations, including the Association of American Medical Colleges (AAMC) [17], the Association of Faculties of Medicine of Canada (AFMC) [15], and the Joint Commission of the Swiss Medical Schools (SMIFK/CIMS) [16], have published sets of EPAs that are intended to cover the full range of UME, including clinical rotations such as clerkships, subinternships, and electives. The numbers of core EPAs (those that are not specialty specific and cover a full UME program) differed between countries (13 published in 2014 in the AAMC-framework, 12 in 2016 in the AFMC-framework, and 9 in 2016 in the SMIFK/CIMS-framework). The wording of some EPA titles was slightly different in the three catalogues. A juxtaposition of the EPAs per framework is shown in Table 2.

Table 2 Entrustable professional activity (EPA) frameworks in undergraduate medical education (UME)
Full size table

EPA development for clinical rotations in UME

We identified 17 articles (47%) that explicitly addressed the development of single EPAs or EPA-based curricula for clinical rotations in UME (Table 1). National frameworks defined core EPAs for UME but did not specify what each specialty should cover in their clinical rotation—if they offered one. Several articles reported on ‘nested’ EPAs (smaller units of clinical activities that fed into core EPAs) or specialty-specific EPAs that could fit into both national frameworks and specialty-specific clinical learning environments [18,19,20,21].

Examples include “Evaluation of patients with respiratory insufficiency” [18] or “Gather a medical history, perform a physical exam and provide a structured summary of the results” [20]. Educators adjusted either nested or specifically developed EPAs for their local clinical rotation context [22,23,24,25,26]. All of these subsets had fewer EPAs in comparison to the national EPA-frameworks in UME.

The degree of referencing existing, relevant documents and validating EPA content, construct, and applicability of newly developed EPAs to specific clinical rotation contexts was varied. Several methods, such as the Delphi method [23], surveys [22, 23] and stakeholder interviews [10, 20] were described to increase the validity of identified clerkship EPAs, but they were not used consistently in the literature.

EPA-based curriculum implementation

Eighteen of the 36 included articles reported on EPA-based curriculum implementation strategies in the context of clinical rotations (Table 1). These were reported at the micro-level within a clerkship curriculum, as well as higher-level implementations that covered several specialties or two educational phases (undergraduate and graduate). We found a wide range of didactic strategies used for teaching EPAs [27,28,29,30], including lectures, small-group discussions, readings, teaching rounds, and online-based learning. In terms of the implementation processes of EPA-based clinical rotations, we did not find evidence for existing best practices in the early phase that would allow for recommendations of specific curricular elements and didactic strategies per core EPA.

Several educational case reports were published that shared institution-based experiences concerning faculty development [24, 31], implemented teaching sessions [29], written or electronic EPA portfolios [32], clerkship curriculum stewardship [33], and working with clinical competency committees [30, 34, 35]. Implementing nested EPAs was one strategy used within clinical rotation contexts [34]. The implementation of longitudinal EPA-based curricula was reported to require more time and personnel resources to synthesize entrustment decisions for summative purposes [36]. This was also the case for time-variable learner handovers between educational phases and workplaces [30].

Longitudinally integrated clerkship structures appeared to be better suited for meaningfully collecting several entrustment data points for high-stakes entrustment decisions over time [18, 29, 30, 33, 37]. Traditional block clerkship curricula ranging from two to 12 weeks were, however, the predominant structure in the EPA-based clerkship curricula [20, 22,23,24,25,26,27, 32, 36, 38,39,40,41,42,43,44]. Two articles reported highly integrated, and longitudinally and specialty oriented, EPA-based curricula that covered both UME and GME, one in pediatrics [35] and one in internal medicine [45]. Only one article reported on the implementation of a curriculum with time-variable progression to residency (shorter or longer duration of undergraduate training based on summative entrustment decisions) [35].

In terms of implementing EPA-based clinical rotation curricula, medical students were reported to act as important change agents in the early implementation phase [29, 30]. Medical students have managed to interest their clinical preceptors in EPA-based curricula and have informed them, in a bottom-up fashion, about reformed curriculum structures. This has helped to generate buy-ins of teaching faculty. One qualitative study reported on the perceptions of teaching faculty in regard to implementing an EPA-based clinical rotation curriculum and found that faculty perceived it as potentially beneficial for patient safety through a better structure of clinical training curricula [44].

EPA assessment in UME clinical rotations

Nineteen of the 36 included articles reported on the assessments of EPAs in clinical rotations in UME (Table 1). We found a wide range of methods, tools, and measures to assess EPAs or aspects of EPAs in terms of knowledge, skills, or attitudes.

Entrustment ratings and summative assessments

Educators used a variety of entrustment scales (binary [32] or different subdivisions of entrustment levels [22]) and varying target entrustment levels for clerkship students (e.g., co-activity versus indirect supervision). Educators typically adjusted published entrustment scales and rubrics [10, 34] to their local needs [23, 25]. Entrustment scales were reduced to binary entrustment decisions [32, 40] or expanded to scales subdivided into 3 to 9 levels of entrustment stages [22, 25, 46] to assess entrustability. Some studies used checklists or rating scales to assess levels of competence, but they did not report any assessment of levels of entrustability [27, 28, 38, 41]. Student self-ratings on performance scales in clinical rotations were compared to expert ratings and did not correlate well [46, 47]. Students tended to overestimate their achieved performance level in comparison to the rating given by experts. One study that compared expert ratings for one EPA did not show significant differences between entrustment ratings of experts [48].

Comparisons of self-entrustment ratings before and after clerkships were also used to identify EPAs that are frequently observed [32, 43, 46]. One article reported on a feasible assessment program that allowed for making high-stakes entrustment decisions and time-variable progress to residency [30]. Another study reported on validating an applied entrustment scale with an educational utility framework based on measures of feasibility, reliability, validity, and educational value [25].

Feedback on EPAs and formative assessments

An EPA-based assessment strategy within a surgical clerkship showed positive effects in the observation frequency of clinical activities and constructive feedback. The use of a binary entrustment scale had no educational effect [32]. In a study across several acute care specialties, feedback on EPAs was perceived as helpful in aligning educational activities. However, students perceived entrustment ratings as less relevant for acute-care settings, since those activities were usually carried out by residents [29]. One study provided evidence concerning the elements of high-quality feedback in the context of EPA-based clinical rotation curricula [39]. These included focused instructions for improvements based on both students’ self-reflection and expert opinion. We found no empirical studies that compared educational outcomes based on entrustment ratings or formative feedback on EPAs as compared to other assessment methods.

Discussion

Summary of the main results

The present scoping review summarizes the emerging evidence concerning EPAs in UME clinical rotations. In total, 36 articles, including three national core EPA frameworks, were identified and used for full data extraction. We saw an upward and international trend in the quantity of published articles. Methods for developing EPAs, implementing EPA-based clinical curricula in UME, and assessing EPAs varied and no clear standard has yet emerged for UME clinical rotations. When developing EPAs, educators should critically consider existing national frameworks, previously published, specialty-specific EPAs, or using some form of a validation process (e.g., survey, Delphi study, stakeholder interviews) for novel EPAs. The EQual rubric has been identified as a helpful tool to ensure the quality of EPA definitions in UME [49, 50].

In terms of EPA-based curriculum implementation, we did not find a gold standard. EPAs used in clinical UME differed significantly in terms of acuity, complexity, and variability both within national frameworks and between specialties (e.g., taking a psychiatric history versus managing a life-threatening emergency). As developing high-quality EPAs is only one element in the design of a clinical curriculum (mainly the goals and objectives in Kern’s six-step approach [51]), a better understanding of the effectiveness of educational and implementation strategies per core EPA is necessary.

Similarly, assessment tools and strategies for (core) EPAs in clinical UME have been used heterogeneously. We found a range of workplace-based assessments and novel entrustment-supervision scales for assessing EPAs in UME clinical training, but these might need to undergo more robust validation in light of current assessment validity frameworks [52, 53]. Overall, the nature and use of entrustment-supervision scales in undergraduate clinical training merits further exploration before recommendations can be made [54].

National EPA-frameworks for clinical curriculum design in UME

Three national EPA-frameworks for UME have been published [14,15,16] and used for clerkship curriculum designs in various medical specialties (Table 1).

Despite some differences in wording and the number of core EPAs, these national frameworks help to compare national educational contexts and could provide a basis for future educational collaborations and international research projects. More research is needed before making recommendations on how to design EPA-based clinical curricula in a way that fits into national frameworks, as well as into specialty-specific contexts.

EPA development for clinical rotations in UME

One challenge when designing clinical curricula in UME is to balance specialty-specific EPAs for undergraduate students planning to choose that specialty and core EPAs that are relevant for all medical students regardless of their chosen specialty. Different approaches to developing EPA-based curricula have been described [55, 56] for GME and could be similarly used for undergraduate clinical rotations. Data from clerkship studies [32, 47] indicate that competency progression changes differently for core EPAs. This is not surprising given the obvious differences between a clinical rotation in surgery as compared to psychiatry. However, these differences should be used to focus and align limited educational resources according to the specific clinical learning contexts instead of trying to accommodate all core EPAs in all clinical rotations. Therefore, we suggest that educators consider three stakeholder perspectives when developing EPA-based curricula for their clinical rotation: the overarching UME curriculum based on core EPAs that are not specialty specific, the specialty-specific clinical context at the level of UME, and the specialty-specific clinical context at the level of GME (residents and attendings as supervisors of medical students). Finally, new tools to evaluate the construct quality of EPAs such as the EQual score [57] should be used in UME.

EPA-based curriculum implementation

From an implementation perspective, we found that faculty development plays an important role [18, 29,30,31]. Educators must be prepared to use teaching and coaching strategies that help trainees move up the entrustment scale [58]. One novel approach would be to work with medical students as part of faculty development [29]. Students have functioned as additional change agents through ad hoc explanations of EPA concepts to clinical preceptors. To implement EPA-based curricula across institutions, we see a need for a more consequential use of published EPA templates [59] that would allow for the transfer of successful implementation strategies. Furthermore, it might be challenging to effectively implement EPA-based clinical curricula together with teaching additional core skills such as time-management skills and personal well-being in clinical workplaces within 4 weeks [26]. Therefore, an integrated longitudinal clerkship curriculum design might offer more opportunities to accommodate these personal development objectives [36].

EPA assessment in clinical rotations in UME

There was an overall heterogeneity with regard to reported educational outcome measures for EPA-based clinical curricula. Assessment methods included written knowledge-based tests [41, 48], a combination of these tests with workplace-based assessments [42], and summative and time-variable entrustment and progression decisions [30]. The articles we found addressed the assessment of EPAs primarily within a clinical rotation. Educators might consider an assessment framework that serves both the specific clinical rotation and an overarching programmatic assessment framework [8] to allow for summative entrustment decisions.

Based on our review results, the potential of (self-) assessing learning goals in the form of EPAs with entrustment-supervision levels is not clear yet. Binary entrustment scales might not yield meaningful assessment data in the context of clinical rotations [32]. The inaccuracy of the self-assessment of competence is well described [60] and has been confirmed in the context of EPAs [46, 47]. However, looking at entrustment estimation from a self-efficacy theory perspective [61] might help educators to address identified needs for more supervision per core EPA to coach students along their clinical learning trajectory.

Limitations

This scoping review has some limitations. Due to the early phase of this emerging research field and the heterogeneity of studies, we were unable to pool data and perform a meta-analysis. Furthermore, since we limited our search to the English phrase “Entrustable professional activity” we may have missed articles from countries that use different terms (e.g., ‘Actividades Profesionales Confiables (APROC)’ in Spanish or ‘Anvertraubare professionelle Tätigkeiten (APTs)’ in German). Additionally, we searched nine databases but did not search the grey literature; thus, we might have missed relevant articles. In terms of data extraction, we focused on aspects relevant to clinical educators; therefore, other stakeholder perspectives might be underrepresented.

Implications for practice and research

EPAs can be meaningfully used to redesign clinical curricula in UME and to focus limited educational resources on competency-based learning outcomes. Strategies to develop validated EPAs in clinical UME and tools for workplace-based assessment of EPAs are emerging. Clinical educators can build on existing national frameworks and specialty-specific EPAs to develop local curricula. Furthermore, EQual rubrics can be used for quality assurance for identifying and developing suitable EPAs [57]. High-quality EPAs should then be used as a basis for systematic clinical curriculum development [51]. A better understanding of EPA-assessment within and across clinical rotations—ideally in the context of programmatic assessment systems—is needed [8].

Availability of data and materials

The datasets used and analyzed for this study are available from the corresponding author on reasonable request.

Abbreviations

AAMC:

Association of American Medical Colleges

AFMC:

Association of Faculties of Medicine of Canada

EPA:

Entrustable Professional Activity

UME:

Undergraduate Medical Education

GME:

Graduate Medical Education

PROFILES:

Principal Relevant Objectives and Framework for Integrated Learning Education in Switzerland

References

  1. O’Dowd E, Lydon S, O'Connor P, Madden C, Byrne D. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011–2018. Med Educ. 2019;53(3):234–49. https://doi.org/10.1111/medu.13792.

    Article  Google Scholar 

  2. Meyer EG, Chen HC, Uijtdehaage S, Durning SJ, Maggio LA. Scoping review of Entrustable professional activities in undergraduate medical education. Acad Med. 2019;94(7):1040–9. https://doi.org/10.1097/ACM.0000000000002735.

    Article  Google Scholar 

  3. Dornan T. Workplace learning. Perspect Med Educ. 2012;1(1):15–23. https://doi.org/10.1007/s40037-012-0005-4.

    Article  Google Scholar 

  4. Hirsh D, et al. “Continuity” as an organizing principle for clinical education reform. N Engl J Med. 2007;356(8):858–66. https://doi.org/10.1056/NEJMsb061660.

    Article  Google Scholar 

  5. AAMC. Number of Medical Schools Offering Longtiudinal Integrated Clerkships. 2018; Available from: www.aamc.org/initiatives/cir/402412/01a.html. [cited 2018 21st October 2018]

    Google Scholar 

  6. Norcini, J. And V. Burch, Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 2007;29:855–871, 9-10, doi: https://doi.org/10.1080/01421590701775453.

    Article  Google Scholar 

  7. Norcini J, Anderson MB, Bollela V, Burch V, Costa MJ, Duvivier R, Hays R, Palacios Mackay MF, Roberts T, Swanson D. 2018 consensus framework for good assessment. Med Teach. 2018;40(11):1102–9. https://doi.org/10.1080/0142159X.2018.1500016.

    Article  Google Scholar 

  8. van der Vleuten CP, et al. A model for programmatic assessment fit for purpose. Med Teach. 2012;34(3):205–14. https://doi.org/10.3109/0142159X.2012.652239.

    Article  Google Scholar 

  9. Shorey S, Lau TC, Lau ST, Ang E. Entrustable professional activities in health care education: a scoping review. Med Educ. 2019;53(8):766–77. https://doi.org/10.1111/medu.13879.

    Article  Google Scholar 

  10. Chen CH, et al. Developing Entrustable professional activities for entry into clerkship. Acad Med. 2016;91(2):247–55. https://doi.org/10.1097/ACM.0000000000000988.

    Article  Google Scholar 

  11. Schatte D, Gavero G, Thomas L, Kovach J. Field guide to boot camp curriculum development. Acad Psychiatry. 2019;43(2):224–9. https://doi.org/10.1007/s40596-018-0933-3.

    Article  Google Scholar 

  12. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, Hempel S, Akl EA, Chang C, McGowan J, Stewart L, Hartling L, Aldcroft A, Wilson MG, Garritty C, Lewin S, Godfrey CM, Macdonald MT, Langlois EV, Soares-Weiser K, Moriarty J, Clifford T, Tunçalp Ö, Straus SE. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73. https://doi.org/10.7326/M18-0850.

    Article  Google Scholar 

  13. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Systematic reviews. 2016;5(1):210. https://doi.org/10.1186/s13643-016-0384-4.

    Article  Google Scholar 

  14. SIWF. Schweizerisches Institut für ärztliche Weiter- und Fortbildung Lernzielkatalog 2020; Available from: https://www.siwf.ch/files/pdf17/allg_lz_d.pdf. [cited 2020 June 2nd, 2020]

  15. AFMC. Association of Faculties of Medicine of Canada. Entrustable professional activities for the transition from Medical School to Residency 2016; Available from: https://afmc.ca/sites/default/files/documents/AFMC_Entrustable_Professional_Activities_EN_0.pdf. [cited 2019 January 1st, 2019]

  16. Michaud P-A, Jucker-Kupper P. PROFILES; Principal Relevant Objectives and Framework for Integrated Learning and Education in Switzerland. In: The Profiles Working Group. Bern: Joint Commission of the Swiss Medical Schools; 2017.

    Google Scholar 

  17. AAMC. The Core Entrustable Professional Activities (EPAs) for Entering Residency. 2018; Available from: https://www.aamc.org/what-we-do/mission-areas/medical-education/cbme/core-epas. [cited 2018 21st October 2018]

    Google Scholar 

  18. Jonker G, et al. Connecting undergraduate and postgraduate medical education through an elective EPA-based transitional year in acute care: an early project report. GMS J Med Educ. 2017;34(5):Doc64.

  19. Menezes N, Hawa R, Oswald R, Lee EK. Does one size truly fit all? The COUPE undergraduate perspective on competency-based medical education in psychiatry. Can J Psychiatr. 2018;63(6):356–60. https://doi.org/10.1177/0706743718758967.

    Article  Google Scholar 

  20. Holzhausen Y, et al. Development of Entrustable professional activities for entry into residency at the Charité Berlin. GMS J Med Educ. 2019;36(1):Doc5.

  21. Horak H, Englander R, Barratt D, Kraakevik J, Soni M, Tiryaki E, UES subcommittee of the AAN. Entrustable professional activities: a useful concept for neurology education. Neurology. 2018;90(7):326–32. https://doi.org/10.1212/WNL.0000000000004947.

    Article  Google Scholar 

  22. Fazio SB, Ledford CH, Aronowitz PB, Chheda SG, Choe JH, Call SA, Gitlin SD, Muntz M, Nixon LJ, Pereira AG, Ragsdale JW, Stewart EA, Hauer KE. Competency-based medical education in the internal medicine clerkship: a report from the Alliance for academic internal medicine undergraduate medical education task force. Acad Med. 2018;93(3):421–7. https://doi.org/10.1097/ACM.0000000000001896.

    Article  Google Scholar 

  23. Hamui-Sutton A, Monterrosas-Rojas AM, Ortiz-Montalvo A, Flores-Morones F, Torruco-García U, Navarrete-Martínez A, Arrioja-Guerrero A. Specific entrustable professional activities for undergraduate medical internships: a method compatible with the academic curriculum. BMC Med Educ. 2017;17(1):143. https://doi.org/10.1186/s12909-017-0980-6.

    Article  Google Scholar 

  24. Keeley MG, Gusic ME, Morgan HK, Aagaard EM, Santen SA. Moving toward summative competency assessment to individualize the Postclerkship phase. Acad Med. 2019;94(12):1858–64. https://doi.org/10.1097/ACM.0000000000002830.

    Article  Google Scholar 

  25. Peters H, Holzhausen Y, Maaz A, Driessen E, Czeskleba A. Introducing an assessment tool based on a full set of end-of-training EPAs to capture the workplace performance of final-year medical students. BMC Med Educ. 2019;19(1):207. https://doi.org/10.1186/s12909-019-1600-4.

    Article  Google Scholar 

  26. Vu TR, Ferris AH, Sweet ML, Angus SV, Ismail NJ, Stewart E, Appelbaum JS, Kwan B. The new internal medicine subinternship curriculum guide: a report from the Alliance for academic internal medicine. J Gen Intern Med. 2019;34(7):1342–7. https://doi.org/10.1007/s11606-019-04957-0.

    Article  Google Scholar 

  27. Jablonover RS, Stagnaro-Green A. ECG as an Entrustable Professional Activity: CDIM Survey Results, ECG Teaching and Assessment in the Third Year. Am J Med. 2016;129:226–30.e1.

    Article  Google Scholar 

  28. Reyes JA, Greenberg L, Amdur R, Gehring J, Lesky LG. Effect of handoff skills training for students during the medicine clerkship: a quasi-randomized study. Adv Health Sci Educ Theory Pract. 2016;21(1):163–73. https://doi.org/10.1007/s10459-015-9621-1.

    Article  Google Scholar 

  29. Jonker G, Booij E, Otte WR, Vlijm CME, ten Cate O, Hoff RG. An elective entrustable professional activity-based thematic final medical school year: an appreciative inquiry study among students, graduates, and supervisors. Adv Med Educ Pract. 2018;9:837–45. https://doi.org/10.2147/AMEP.S176649.

    Article  Google Scholar 

  30. Andrews JSMD, et al. Education in pediatrics across the continuum (EPAC): first steps toward realizing the dream of competency-based education. Acad Med. 2018;93(3):414–20. https://doi.org/10.1097/ACM.0000000000002020.

    Article  Google Scholar 

  31. Lupi CS, Ownby AR, Jokela JA, Cutrer WB, Thompson-Busch AK, Catallozzi M, Noble JM, Amiel JM, Association of American Medical Colleges Core Entrustable Professional Activities for Entering Residency Faculty Development Concept Group. Faculty development revisited: a systems-based view of stakeholder development to meet the demands of Entrustable professional activity implementation. Acad Med. 2018;93(10):1472–9. https://doi.org/10.1097/ACM.0000000000002297.

    Article  Google Scholar 

  32. Curran VR, Deacon D, Schulz H, Stringer K, Stone CN, Duggan N, Coombs-Thorne H. Evaluation of the characteristics of a workplace assessment form to assess Entrustable professional activities (EPAs) in an undergraduate surgery Core clerkship. J Surg Educ. 2018;75(5):1211–22. https://doi.org/10.1016/j.jsurg.2018.02.013.

    Article  Google Scholar 

  33. Hirsh D, Holmboe E, ten Cate O. Time to trust: longitudinal integrated clerkships and entrustable professional activities. Acad Med. 2014;89(2):201–4. https://doi.org/10.1097/ACM.0000000000000111.

    Article  Google Scholar 

  34. Ten Cate O, et al. The EPA-based Utrecht undergraduate clinical curriculum: development and implementation. Med Teach. 2018;40(5):506–13. https://doi.org/10.1080/0142159X.2018.1435856.

    Article  Google Scholar 

  35. Murray KE, et al. Crossing the gap: using competency-based assessment to determine whether learners are ready for the undergraduate-to-graduate transition. Acad Med. 2019;94(3):338–45.

  36. Evans DB, et al. Continuity in undergraduate medical education: Mission not accomplished. J Gen Intern Med. 2019;4(10):2254–9.

  37. Pereira AG, Woods M, Olson APJ, van den Hoogenhof S, Duffy BL, Englander R. Criterion-based assessment in a norm-based world: how can we move past grades? Acad Med. 2018;93(4):560–4. https://doi.org/10.1097/ACM.0000000000001939.

    Article  Google Scholar 

  38. Norbury JW, et al. Impact of a revised curriculum focusing on clinical neurology and musculoskeletal care on a required fourth-year medical student physical medicine and rehabilitation clerkship. Rehabil Res Pract. 2016;2016:6197961.

    Google Scholar 

  39. Duijn C, et al. Am I ready for it? Students’ perceptions of meaningful feedback on entrustable professional activities. Perspect Med Educ. 2017;6(4):256–64. https://doi.org/10.1007/s40037-017-0361-1.

    Article  Google Scholar 

  40. Thompson LR, et al. Development of an assessment for Entrustable professional activity (EPA) 10: emergent patient management. West J Emerg Med. 2017;18(1):35–42. https://doi.org/10.5811/westjem.2016.10.31479.

    Article  Google Scholar 

  41. Kukulski P, Ward M, Carter K. Ultrasound for volume assessment in patients with shock: effectiveness of an educational intervention for fourth-year medical students. Cureus. 2018;10:e2129.

    Google Scholar 

  42. McClintic JA, Snyder CL, Brown KM. Curricular innovation in the surgery clerkship: can assessment methods influence development of critical thinking and clinical skills? J Surg Educ. 2018;75(5):1236–44. https://doi.org/10.1016/j.jsurg.2018.02.012.

    Article  Google Scholar 

  43. Colbert-Getz JM, et al. To what degree are the 13 Entrustable professional activities already incorporated into Physicians’ performance schemas for medical students? Teach Learn Med. 2019;31(4):361–9.

  44. Czeskleba A, Holzhausen Y, Peters H. Patient safety during final-year clerkships: A qualitative study of possible error sources and of the potential of Entrustable Professional Activities. GMS J Med Educ. 2019;36:Doc18.

    Google Scholar 

  45. Elnicki MD, et al. An Entrustable professional activity (EPA)-based framework to prepare fourth-year medical students for internal medicine careers. J Gen Intern Med. 2017;32(11):1255–60. https://doi.org/10.1007/s11606-017-4089-8.

    Article  Google Scholar 

  46. Klapheke M, Johnson T, Cubero M. Assessing Entrustable professional activities during the psychiatry clerkship. Acad Psychiatry. 2017;41(3):345–9. https://doi.org/10.1007/s40596-017-0665-9.

    Article  Google Scholar 

  47. Klapheke M, Johnson T, Cubero M. Third-and fourth-year medical student self-assessment in entrustable professional activities. Med Sci Educ. 2017;27(2):159–64. https://doi.org/10.1007/s40670-017-0377-5.

    Article  Google Scholar 

  48. Ming D, et al. Discharge summary training curriculum: a novel approach to training medical students how to write effective discharge summaries. Clin Teach. 2018.

  49. Meyer EG, Taylor DR, Uijtdehaage S, Durning SJ. EQual rubric evaluation of the Association of American Medical Colleges’ core entrustable professional activities for entering residency. Acad Med. 2020;95(11):1755–62. https://doi.org/10.1097/ACM.0000000000003504.

    Article  Google Scholar 

  50. Ten Cate O, Taylor DR. The recommended description of an entrustable professional activity: AMEE Guide No. 140. Med Teach. 2020:1–9.

  51. Thomas, P.A., et al., Curriculum development for medical education: a six-step approach. Baltimore: Johns Hopkins University Press; 2016.

  52. Messick S. Standards of validity and the validity of standards in performance asessment. Educ Meas Issues Pract. 1995;14:5–8.

    Article  Google Scholar 

  53. Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to K ane's framework. Med Educ. 2015;49(6):560–75. https://doi.org/10.1111/medu.12678.

    Article  Google Scholar 

  54. Ten Cate O, Schwartz A, Chen HC. Assessing trainees and making entrustment decisions: on the nature and use of entrustment-supervision scales. Acad Med. 2020;95(11):1662–9. https://doi.org/10.1097/ACM.0000000000003427.

    Article  Google Scholar 

  55. Kwan J, Crampton R, Mogensen LL, Weaver R, van der Vleuten CPM, Hu WCY. Bridging the gap: a five stage approach for developing specialty-specific entrustable professional activities. BMC Med Educ. 2016;16(1):117. https://doi.org/10.1186/s12909-016-0637-x.

    Article  Google Scholar 

  56. Boyce P, Spratt C, Davies M, McEvoy P. Using entrustable professional activities to guide curriculum development in psychiatry training. BMC Med Educ. 2011;11(1). https://doi.org/10.1186/1472-6920-11-96.

  57. Taylor DR, Park YS, Egan R, Chan MK, Karpinski J, Touchie C, Snell LS, Tekian A. EQual, a novel rubric to evaluate entrustable professional activities for quality and structure. Acad Med. 2017;92:S110–S7, 11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions. https://doi.org/10.1097/ACM.0000000000001908.

    Article  Google Scholar 

  58. Barrett J, Trumble SC, McColl G. Novice students navigating the clinical environment in an early medical clerkship. Med Educ. 2017;51(10):1014–24. https://doi.org/10.1111/medu.13357.

    Article  Google Scholar 

  59. Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013;5(1):157–8. https://doi.org/10.4300/JGME-D-12-00380.1.

    Article  Google Scholar 

  60. Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med. 2005;80(Supplement):S46–54. https://doi.org/10.1097/00001888-200510001-00015.

    Article  Google Scholar 

  61. Sagasser MH, Kramer AWM, Fluit CRMG, van Weel C, van der Vleuten CPM. Self-entrustment: how trainees’ self-regulated learning supports participation in the workplace. Adv Health Sci Educ. 2017;22(4):931–49. https://doi.org/10.1007/s10459-016-9723-4.

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful for the feedback and advice of Olle ten Cate on earlier versions of the manuscript.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. University Hospital of Old Age Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland

    Severin Pinilla, Eric Lenouvel & Stefan Klöppel

  2. Department for Assessment and Evaluation, Institute for Medical Education, University of Bern, Bern, Switzerland

    Severin Pinilla & Sören Huwendiek

  3. University Hospital of Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland

    Andrea Cantisani, Werner Strik & Christoph Nissen

Authors
  1. Severin Pinilla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric Lenouvel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Cantisani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefan Klöppel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Werner Strik
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sören Huwendiek
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Christoph Nissen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SP and EL conceptualized the study. SP, AC, WS, SK, SH, and CN were responsible for the initial draft of the manuscript. All authors participated in consensus processes and drafted and revised the final manuscript. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Severin Pinilla.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

All authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplemental Table 1

. Inclusion and exclusion criteria.

Additional file 2: Supplemental Table 2

. Applied codes for data extraction.

Additional file 3: Supplemental Table 3

. Data extraction EPAs in clerkships.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pinilla, S., Lenouvel, E., Cantisani, A. et al. Working with entrustable professional activities in clinical education in undergraduate medical education: a scoping review. BMC Med Educ 21, 172 (2021). https://doi.org/10.1186/s12909-021-02608-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12909-021-02608-9

Keywords

BMC Medical Education

ISSN: 1472-6920

Contact us