Medical educators in North America are embracing competency-based medical education (CBME) as an opportunity to improve the education of our learners and influence patient care.1, 2 One innovation of CBME is Entrustable Professional Activities (EPAs): the specialty-specific clinical tasks which can be fully entrusted to a learner (i.e., the learner can be granted unsupervised responsibility) as they demonstrate competence in performing these tasks.3 EPAs offer the potential of an efficient approach to assessing multiple competencies in the context of authentic clinical work. However, as front-line internists, clinical educators, and education researchers, we recognize that our current implementation of entrustability assessments within internal medicine training does not seamlessly translate to meaningful learner assessment. We bring to these Perspectives a shared concern that EPAs and their associated assessments require additional thought and attention in order to incite significant educational change in internal medicine.

In-the-moment, workplace-based assessment informed by direct observation and progressive entrustment of the learner are cornerstones of an assessment program using EPAs and were first developed for assessment in the procedural-based specialty of obstetrics-gynecology.2 However, many of the tasks of internal medicine are less well-circumscribed than the task of observing a procedure, much learner activity is closely monitored but not directly observed, and the outcome of the learner’s work (i.e., the patient outcome) may take more time to unfold than procedural outcomes. Perhaps anticipating these challenges, internal medicine has been at the forefront of research into direct observation, entrustment, and clinical supervision.4,5,6,7 In these Perspectives, we outline important concerns with ad hoc (i.e., in-the-moment) entrustability assessments within internal medicine post-graduate CBME training and suggest potential solutions.

THE CHALLENGES OF AD HOC ENTRUSTABILITY ASSESSMENTS IN INTERNAL MEDICINE

Entrustability assessments implemented in procedural-based specialties, within a culture of direct observation and feedback, have rested on the question “Can I leave my resident alone to do this task?”.[8, p.187] There is perhaps no better example of entrustment than an attending physician physically handing a scalpel to a resident and leaving the operating room so that the resident performs the procedure unsupervised. This act of entrustment is presumably based on an assessment of the resident’s competence grounded in prior direct observation. The anchors on entrustability rating scales in this context align with the construct of progressive autonomy in performing a procedural skill.8, 9 These construct-aligned entrustability assessments for procedural skills requires fewer observations than with other rating scales.10, 11 This efficiency may be due, in part, to the resident and supervisor sharing an understanding of the well-defined steps of a procedure.

However, many of the affordances present in entrustability assessments of procedural skills are not present in internal medicine training. First, much of internal medicine’s work is non-procedural. A patient’s course in hospital may unfold over days and require iterative data gathering and there may not be one “task” that is observable from beginning to end. In-patient care is provided by inter-professional teams so it is difficult to attribute any task to one provider.12 Resident and supervisor may lack a shared understanding of the task (e.g., does admitting a patient require a focused history and overnight admission orders or a comprehensive history and orders for the next 48 hours?). Moreover, frequent transitions in patient care mean that multiple supervisors may be involved in the patient admission and thus no single supervisor can comment on performance of an EPA related to caring for the patient across the complete hospital admission.13

Second, supervisors in medicine do not naturally use the language of trust or entrustment and, when speaking about their supervisory decisions, are more apt to describe feeling comfortable with or having confidence in a resident performing a task.4, 7 Adding to the language barrier, trust and entrustment are not the same. Trust refers to our confidence or reliance on another person, whereas entrustment relates to the action of granting that trust.14 Entrusting a resident may have more to do with presumptive trust based on the resident’s level of training as opposed to trust grounded on prolonged exposure to and assessment of the resident, particularly when longitudinal relationships between internal medicine residents and supervisors are scarce.13, 14

Third, the issues run deeper than issues of language. Ad hoc entrustability assessments are often promoted as arising opportunistically as EPAs are encountered during clinical supervision and are envisioned as a simple, construct-aligned rating.3 However, the entrustability assessment occurs in a context that is influenced by multiple factors including the supervisor, the resident, the relationship between supervisor and resident, the clinical context, and the patient care task.4 Supervisors’ approaches to supervision are not static but fluidly change in response to these contextual factors and by their prior knowledge of the resident.7 Thus, treating the entrustability assessment as an acontextual, isolated decision about the task under observation is likely misleading.15

Fourth, internal medicine is not a culture rife with direct observation and feedback.16 The ward-based system of learning includes regular periods when clinical supervisors are expected to be absent (e.g., nights and weekends). During these times, it is the system that entrusts trainees. Consider a patient with diabetic ketoacidosis (DKA) admitted to the clinical teaching unit. Overnight, the senior resident may admit the patient without direct supervision. The following morning, the clinical supervisor may see the patient without the resident, review the orders, and check the lab results. Then, when the supervisor and resident discuss the patient, the supervisor may listen to the resident’s plan and make suggestions based on their own clinical assessment. Much of this clinical supervision has occurred as “backstage oversight,” in the absence of the resident and without direct observation.17 This use of backstage oversight means that residents are never unsupervised because the supervisor is in the background, even when physically absent from the ward, ready to intervene if they have concerns.4, 7, 17 Furthermore, in this example, clinical supervision was provided without the supervisor making explicit entrustment decisions. Thus, an entrustability scale anchored from “I had to do” through to “I did not need to be there”8 would not be construct-aligned with the supervisor’s actions.

HOW DO WE MOVE FORWARD?

By attempting to problematize entrustability assessment in internal medicine, we want to foster a productive discussion that leads to implementable solutions.

  1. 1.

    Clarify the purpose of the assessment

As an education community, we should clarify how entrustability assessments can inform CBME.18, 19 Acknowledging that part of the impetus for CBME is to tailor education to the individual resident (by focusing on learning outcomes rather than time spent in training), leads to a conclusion that we should focus our entrustability assessments on tracking the individual resident’s development. Identifying residents who require more support will be a natural by-product of having more detailed assessments of each individual. In order to track the development of a resident’s competence through entrustment decisions, we will need to find strategies to account for the influence of presumptive entrustment and the contextual factors weighing into each ad hoc entrustment decision.

  1. 2.

    Reconsider each of the fundamental concepts of entrustment

  1. a)

    Not all tasks are suitable as EPAs

Many of the daily activities within a specialty are not appropriate as EPAs (i.e., they may not be specific to the specialty, they may not be observable or measurable, etc.)20 and many of the summative EPAs are too broad to serve as the focus for a single daily encounter. Thus, we need to examine the activities that comprise the practice of internal medicine and determine their suitability as EPAs. For tasks that do not typically include direct observation (e.g., admitting a patient), EPAs may be ill-suited as the assessment focus, whereas for tasks where supervisors routinely observe a resident at work (e.g., a family meeting) then EPAs may be fit for purpose.

  1. b)

    Entrustability may not be the “right” unit of measurement to capture daily internal medicine supervisory decisions

Fundamentally, we need to determine whether trust and entrustment are truly at the core of supervision of non-procedural tasks. The relationships between the level of supervision provided, the supervisor’s trust in the resident, the supervisor’s entrustment judgment and determinations of competence are complex and require further examination.4, 7, 17 Mixed methods research could explore current entrustment ratings to unpack with supervisors the meaning that underlies their ratings. Experimental study designs could compare language used for the anchors on rating scales to determine construct alignment. Qualitative research should explore what else is at the core of non-procedural clinical supervision.

  1. c)

    Direct observation and indirect supervision both have a role in ad hoc entrustment

Acknowledging that internal medicine supervisory practices include a significant amount of backstage oversight (indirect supervision), what is essential to directly observe? It seems intuitive that direct observation of and feedback on competencies such as physical examination or communication with patients are needed for identifying otherwise unrecognized learning gaps and to inform teaching and feedback. But ad hoc entrustment decisions informed by multiple pieces of evidence have the potential to be robust without direct observation. Reflecting back on the DKA example, what would we need to observe to significantly change the patient care provided by the resident, compared to our insights based on indirect supervision? We could benefit from more deeply examining when and how direct observation could complement indirect supervision and how both of these supervisory activities could be used for assessment.

  1. 3.

    Examine whether quantitative measurement is the best approach

We encourage critical appraisal of the validity evidence of entrustability scales in internal medicine. Initial evidence demonstrates that residents’ scores improve over time, both within individuals and across training levels.21,22,23 However, given internal medicine’s long familiarity with workplace-based assessments such as the mini-cex, faculty may be using a 5-point rating scale as they always have, irrespective of the construct that is being assessed. We need more robust and diverse validity evidence to determine if entrustment is truly what is being measured.

It may be that a quantitative, psychometric approach to rating entrustment cannot capture what it is we aspire to capture. Known limitations of rating scales will persist. Do we simply have a problem of construct alignment for which further iteration of the rating scale will provide a solution? Even if supervisory encounters can be captured as a single rating, will ad hoc entrustability ratings provide competence committees with the information they need to make defensible decisions about resident progression towards competence?

A shift in focus towards descriptive data, i.e., robust descriptions of the encounter and the judgments (which are being collected but on which little has yet to be published), may get us closer to the heart of what we want to know. There are important resident-supervisor interactions occurring on a regular basis during clinical supervision, and harnessing the essence of these interactions should provide meaningful information about a resident’s competence. We readily acknowledge that this shift is not without challenges: writing detailed narratives will add to a supervisor’s workload, the volume of information the resident and program need to process will increase, and new skills in interpreting qualitative data will be required.24

WE CAN DO THIS

Implementation science informs us that the adoption and integration of new practices is a process requiring a continuous cycle of re-examination, modification, implementation, and re-examination. At this juncture in the implementation of CBME, individual educators, programs, and national bodies have invested enormous effort in order to launch the first iteration of entrustability assessments. As the evidence emerges from these early adopters, it is an ideal moment to pause and examine what that evidence is telling us and where we need to go with our next iteration.25, 26 By being open to this challenge and engaging in the process of continuous improvement, we aim to strengthen the assessments provided to our residents and ultimately improve the quality of patient care that they provide.