A collective case study of supervision and competence judgments on the inpatient internal medicine ward

Drawing inspiration from Kennedy’s method of using observational fieldwork and brief interviews to study clinical supervision [7], assessment of trainees’ competence for independent clinical work [6] and help-seeking by trainees [10], we observed the interactions between supervisor-trainee dyads on inpatient wards. In contrast to the grounded theory methodology used by Kennedy and her team, we employed case-study methodology, which “facilitates exploration of a phenomenon within its context” [11] and is a design that is well-suited to research in complex settings where concepts may not have simple causal relationships [12]. A collective case study combines a set of cases, wherein each case is less important than gaining an understanding of the theoretical underpinnings of a unit of analysis within the cases [11, 13]. We defined each case as one attending-resident dyad who were randomly scheduled by the residency program to work together for a two-week period. We approached the cases using an interpretivist paradigm—taking reality as a social construct arising from how individuals experience the world and interact with one another—which allows for the exploration and deeper understanding of a particular concept within and across the cases while considering the importance of environmental context [12].

The study took place on the internal medicine clinical teaching unit (CTU) inpatient wards at three University of British Columbia (UBC)-associated tertiary care hospitals.

Chief medical residents sent invitations to senior medical residents and attendings scheduled at the three CTU sites. Recruitment targeted attendings who had earned a reputation among their supervisor colleagues (TG, RH) on the research team for ‘empowerment’ and ‘mixed practice’ styles [8] because supervisors using these styles are more likely to adjust their level of supervision based on an assessment of the trainee’s competence than those using ‘minimalist’ and ‘direct care’ styles with unchanging levels of supervision. Attending participants included general internists or geriatricians on the internal medicine service, with experience spanning from a pre-certification junior attending to more than 15 years in practice. Trainee participants included second- and third-year internal medicine residents spanning their first to last rotation as a senior resident. All dyads were unique with residents and attendings recruited to participate in a single dyad to maximize variation in sampling.

We collected data through field observations and semi-structured interviews. Each dyad was observed for up to seven hours per day on two days approximately one week apart. Observations focussed on the senior resident and their typical daily activities on the ward in order to note the resident’s apparent or voiced need for support, the presence or absence of the attending physician, and the interactions between the resident and the attending. The activities included rounding on patients, overseeing the work of junior trainees (medical students and junior residents), meeting with allied health representatives (‘team care’), teaching junior trainees, and reviewing with the attending physician the care decisions for all patients assigned to the team (‘running the list’). Observations were documented in field notes. At the end of the workday, each member of the dyad participated in a separate semi-structured interview. Both were prompted to discuss incidents observed during the day in which attending physicians exhibited supervisory behaviours or might have been expected to engage in supervisory behaviours. Other questions related to the resident’s need for support, and the competence judgments of the attending. Within a few weeks after the second observation day, there was an additional semi-structured interview with the attending physician to discuss any further developments in their work with the senior resident and to review their final assessment of the resident. Interviews were audio-recorded, transcribed, and de-identified.

At the beginning of the research process, TG was a final-year general internal medicine resident in supervised practice. Later in the process, TG was working as a general internist and was at times supervising residents and medical students. TG had a pre-existing relationship with some residents in the study and with all of the attending physicians. AG is a PhD education researcher with expertise in rater cognition. RH is an internal medicine and palliative care physician and education researcher with expertise in feedback and assessment. Our use of an interpretivist paradigm made it important to engage in the process of reflexivity [14]. We aimed to continually maintain consideration of how researcher perspectives and biases could be influencing interpretation of the data [15] by engaging in frequent research meetings, during which we explored how our unique backgrounds influenced our understanding and analysis.

Following collective case study methodology [11, 16], we engaged in a recursive process of examining data to simultaneously summarize and interpret it during its collection to inform modifications to subsequent observations, field notes, and interview questions. Once collected, we analysed the data within each case and then compared across cases. Within-case analysis began with all three researchers independently analysing each dyad’s interview transcripts and field notes to identify incidents involving supervisory behaviours (e.g. attending goes to bedside with resident or attending does not go to team rounds) or incidents in which we might have expected a particular supervisory behaviour (e.g. patient suddenly deteriorates or resident struggles at team rounds). These were discussed in a group meeting, and then one researcher (AG) returned to the data and verified these incidents for accuracy and consistency, compiling all available information from field notes and interview transcripts that pertained to that incident. Through further review and discussion of the compiled information, the research team developed a systematic procedure [13] to organize the information pertaining to each incident by parsing it into information that described the supervisory behaviour, the trigger of the supervisory behaviour, why the attending responded to that particular situation with that particular supervisory behaviour, how the incident informed their judgment of trainee competence, and any impact on subsequent supervision. Such organization enabled the sequence of incidents to be tracked within each dyad and facilitated our examination for any pattern of changes within each dyad, particularly in regard to judgments of the resident’s competence.

For the cross-case analysis, the organization of the incident information allowed for it to be re-sorted to focus on each supervisory behaviour rather than on each dyad. This enabled each supervisory behaviour (e.g. goes to bedside with trainee) to be tracked across incidents in multiple dyads and facilitated our examination of the information for any patterns, in particular the alignment with competence judgments for various residents.

Ethics approval was received from the University of British Columbia behavioural research ethics board (H18-00878). Appropriate institutional approvals were also obtained from the health authorities (Fraser Health, Providence Health Care, and Vancouver Coastal Health) where the observations took place.

We identified eight supervisory behaviours, which are listed in Table 1 (found in the Electronic Supplementary Material [ESM]) with representative examples, through examination of the incidents. The incidents were labelled by the supervisory behaviour that represented a change or a choice in the level of supervision provided to the resident at that moment. More specifically, the supervisory behaviour ‘providing direct patient care’ refers to incidents where the attending delivered care for the patient themselves with the resident present (dyad 63) or not (dyads 15, 20). The supervisory behaviour ‘going to the bedside’ represents an attending’s decision to accompany the resident as they provide patient care (dyads 3, 18, 41, 72, 98). The supervisory behaviour ‘helping to navigate hospital logistics and bureaucracy’ (dyads 15, 24, 41) reflects incidents where the attending led administrative tasks to procure the patient’s access to investigations or resources. The supervisory behaviour ‘taking over the lead’ during group discussions refers to the attending stepping into a leadership role during team care meetings or running the list discussions (dyads 15, 16, 20, 41, 98). The label ‘interjecting to change treatment or management’ (dyad 3) refers to the attending correcting the resident’s orders whereas the label ‘answering trainee questions and helping with decision-making’ (dyads 3, 15, 41, 72) reflects a more supportive discussion of how the resident could proceed. The attending providing a more substantial teaching moment inspired by the immediate incident but focusing on future responsibilities was labelled ‘providing directed teaching’ to the trainee (dyad 63). The supervisory behaviour ‘checking up and checking behind’ referred to the attending monitoring the resident’s performed activities without directly observing them (dyads 3, 16, 20, 63, 98) or while physically present but not directly involved with the activities (dyad 16).

Supervisory behaviours were used in response to factors related to the resident and also for factors unrelated to the residents. The supervisory behaviour ‘going to the bedside’, for example, was used in response to trainee factors, such as their perceived ability, observed performance, and experience with the task. It was used to support the resident when support was needed (dyad 18), to assist the resident after they had asked for help (dyad 72), and as a means to identify a performance gap and then provide feedback to the resident (dyad 98). However, the supervisory behaviour ‘going to the bedside’ was also used when the clinical situation (i.e. non-trainee factors) warranted, such as when the patient was acutely sick or dying (dyads 18, 98), as a means of checking in with nursing staff (dyad 41), and as a routine part of supervision with “zero concern with the [resident]” (dyad 18) and when the resident did not need or ask for support (dyads 15, 98, 41).

Supervisory behaviours were not always informed by judgments of a resident’s competence. When they were, the choice to use a given supervisory behaviour, such as ‘checking up and checking behind’, could be informed by either previous impressive performance (dyads 3, 16, 63) or unimpressive performance (dyad 98). A supervisory behaviour informed by a competence judgment did not necessarily result in the resident receiving the level of supervision they needed, and this could be realized at a later time. For example, “R3 strikes me so far as being capable in terms of a manager, in terms of being sufficiently assertive and able to direct team care” so the attending began to arrive late at team care and then stay in the background before ultimately not attending team care or a running the list session. Although the resident was eventually allowed to do the task without the attending present, when the attending ‘checked up and checked behind’, A3 discovered “a couple days later that maybe there were things that could have been done better”, revealing that the resident did not perform as competently as expected and could have used supervisory support during team care.

A change in the attending’s judgment of the resident’s competence did not always correspond with a change in subsequent supervisory behaviours. In one incident, the attending discovered that R20 had delayed important care while working solo overnight. The attending acknowledged: “I think that there’s some knowledge gaps in the way [the resident] manages things” and that “maybe someone stronger probably would have known”. Thus, the attending’s judgment of the resident’s competence was that “maybe R20’s not quite as strong as [I thought] R20 was before”. However, when asked directly if they changed how they supervised the resident after the incident, A20 clearly responded “no”, and no change was observed.

Similarly, the pattern of supervisory behaviours used within a dyad was not a good indication of the attending’s overall judgment of the resident’s competence. For example, the five incidents of dyad 15 seem to paint a picture of a trainee who repeatedly needed the attending to step in and provide support: the attending ‘took over the lead’ during team care and then 20 minutes later when the resident asked about how to manage a patient’s fitness to drive, the attending ‘helped to navigate bureaucracy’ by taking it on themselves before offering to ‘provide direct patient care’ for other patients while the resident was off the ward for teaching. The next week the attending ‘provided direct patient care’ to a patient with a life-threatening condition that the resident could not handle in the morning and then ‘helped the trainee with decision-making’ for a patient with a rare condition in the afternoon. However, the attending described the resident as “a very strong clinician” and when asked why they used these supervisory behaviours, they said they were helping “in anticipation of the weekend and the limited services available” and for “guidance about how to deal with this quite complicated issue because it’s something that [the resident] may not have seen before.”

When the identified supervisory behaviours were tracked through numerous incidents across multiple dyads, they corresponded with varying assessments of resident competence. For example, the supervisory behaviour ‘answering trainee questions and helping with decision-making’ was used when the resident’s competence with that task ranged from “maybe a minor knowledge gap” (A3) to “understanding was there for R72 but they just maybe lacked the confidence to make that decision on their own” (A72) to “R41 actually is highly detail oriented and keeps excellent notes and knows what’s going on with the patient” (A41). The supervisory behaviours that involved the attending stepping in to provide additional support tended to correspond with them saying that their judgment of resident competence was unaffected or even high for that task. They made statements like this was “a very strong [resident]” (A63) and “easily a task I could have completely delegated to the [resident] to do” (A41) but regardless they stepped in for non-trainee factors such as having the time to help (dyad 15) or to increase efficiency on ward (dyad 41) or it was a very sick patient or unusual diagnosis (dyads 98, 15) or they just happened to be close-by and “thought it could have been a terminal event and I should be there as the attending physician” (A18).