What this paper adds
Feedback is a central topic in medical education that has been deemed essential for helping learners to improve performance and meet standards of competency. In an educational system that strives to focus on both academic milestones and clinical outcomes, patient care has become a prominent feature. It is unclear, however, what is known about feedback to medical learners and its association with patient-related outcomes. Our analysis describes the state of the medical education literature on the topic of feedback to medical learners and its association with characteristics of improved patient care and provides a description of areas in need of further investigation.
Introduction
Feedback has been a focus of the medical education literature for decades, covering a variety of features, techniques and purposes. Feedback has been deemed essential to promoting learning [1], improving performance [2], acquiring clinical skills [3] and more recently, meeting standards of competency [4]. Feedback is an important component of formative assessment (the exchange of information designed to support development) [5]. The outcome of a summative assessment that measures success or proficiency can also be used as feedback to guide subsequent efforts [6]. Despite the importance of feedback and the attention it has received in scholarly literature, effective feedback remains difficult to achieve within the context of clinical education [5].
Many definitions of feedback exist in the literature, most focusing on behavioural change and performance improvement. A systematic review of feedback definitions in the medical literature by Van der Ridder [7] yielded the following compilation: “Feedback in clinical education is specific information about the comparison between a trainee’s observed performance and a standard, given with the intent to improve the trainee’s performance”. Is the focus of feedback as a means to effecting behavioural change enough? As recognized by Ende in his 1983 sentinel article on the use of feedback in medical education, “The goal of clinical training is expertise in the care of patients” [8]. Should there be a more robust feedback end goal of improved patient care and outcomes?
Improving the patient experience of care, as well as improving the health of populations and reducing healthcare costs, are core dimensions of the Triple Aim as outlined in 2008 by the Institute for Healthcare Improvement [9]. In 2012, the Accreditation Council for Graduate Medical Education responded with the Next Accreditation System [10]. This shift toward competency-based medical training through the use of educational milestones was designed as a way to focus the system on both educational and clinical outcomes. The topic of patient care features prominently in the milestone sub-competencies specific to each residency specialty [11] and includes well-defined patient outcomes. Assessing these outcomes will become increasingly important as healthcare moves toward a value-based system [12]. Effective feedback strategies are believed to be crucial to help medical learners achieve these outcomes [3]. Programs that provide clinical performance feedback to promote compliance with patient care-related quality measures are now being utilized at the practising physician level [13]. It is unclear, however, what is known about feedback and improved patient outcomes at the medical learner level. To that end, we sought to answer the question “What is the association of medical learner feedback with characteristics of improved patient care?”
A 2016 scoping review conducted by the authors [14] provided a broad map of the feedback literature for undergraduate and graduate medical learners. Scoping reviews are utilized to examine the extent of research activity in a topic area, determine the value of undertaking a full systematic review and summarize research findings, as well as identify gaps for further research focus as is the purpose of this study [15]. For the current analysis, a subset of the articles was analyzed utilizing an evidence synthesis mapping process [16, 17] to explore the relationship of medical learner feedback to patient management and outcomes.
Methods
The data sources for the initial scoping review have been described in full previously [14]. In brief, six databases were searched (Ovid MEDLINE, CINAHL, ERIC, ProQuest Dissertations and Theses Global, Scopus, and Web of Science) as well as seven well-known medical education journals and the reference lists of 11 key articles on feedback. The search terms feedback; feedback; psychological; medical students; assessment; self-assessment; internship and residency; resident; fellows; medical education; faculty; faculty; medical; and reflection were utilized. The search was limited to articles published from 1 January 1980 to 31 December 2015. A data extraction form was completed that included components such as methodology, intervention type, focus of feedback content and outcome measures. Twenty-seven articles, a small segment of the total 650, were identified from that review by two of the authors (V. H., R. B.) who utilized patient data or chart reviews as outcomes measures to assess the clinical impact of the feedback interventions.
The authors developed a data extraction form based on review models found in the medical literature [17, 18]. Two of us (V. H., R. B.) reviewed all the studies independently and conducted the data extraction component. A third author (K. V.) reviewed and resolved any discrepancies noted.
A follow-up search was conducted on 27 March 2017 to assess for new relevant articles. The prior search terms were utilized with the addition of chart reviews and patient outcomes for the years 2016–2018 (to enable capture of articles published ahead of print). This search yielded 119 articles which were then assessed for inclusion by two of the authors (V. H., R. B.). No additional articles were found that utilized patient data to assess the impact of feedback interventions.
Results
A total of 27 articles focused on feedback in medical learners and patient-related outcomes, with years of publication ranging from 1980 to 2015, were reviewed in-depth (see Table 1 of the online Electronic Supplementary Material (ESM)) [19‐45].
Patient-related outcomes
The vast majority of the studies reported that the feedback intervention had a positive effect on patient-related outcomes (Table 2 of online ESM), while the remainder reported no change. The 21 studies with positive results centred mostly on improved documentation in areas such as discharge summaries, patient progress notes, prescription writing, and diagnostic capture and coding. Other areas that yielded positive results included patient management skills and patient satisfaction scores. Of the few studies that did not show a positive effect (Table 2 of online ESM), two were related to adherence to practice guidelines with individual studies on adenoma detection rate, immunization rates and obtaining a psychosocial history. There were no commonalities in the group that did not show improvement related to the type of patient outcomes or feedback delivery method.
The most common patient-related outcomes (Table 3 of online ESM) were in the realm of patient management, such as adherence to practice guidelines and immunization rates, and documentation in the areas of medical record-keeping, discharge summaries and diagnostic coding. The patient-related data that were utilized to assess the impact of the feedback interventions most commonly consisted of practice performance parameters such as adherence to practice guidelines and immunization rates. Procedural or exam skills were also frequently assessed and included components such as measurement of liver span, Papanicolaou smear adequacy or adenoma detection rates. Patient-related data were primarily collected through chart review with utilization of a checklist or chart audit tool or by reviewing skills data such as pap smear adequacy. Feedback data were delivered to the participants most frequently through a written format. If the feedback was delivered in-person, the majority involved the authors or another faculty member.
Participants and setting
The majority of the articles involved one medical discipline at one institution (Table 4 of online ESM). The remaining articles described either medical students at one school, one discipline at multiple institutions or multiple disciplines at one institution. The studies primarily involved internal medicine residents alone and medical students, with the remainder utilizing a variety of other resident specialties. The four studies involving medical students focused on physical exam skills, interviewing techniques and written documentation and all had positive results. There were no significant differences noted between the studies involving students and more advanced trainees. Gastroenterology fellows were utilized in one study while the participants were not clearly identified in the remaining study.
The study settings were primarily academic teaching hospitals with two at Veterans Affairs (VA) teaching hospitals and one at an outpatient multispecialty group practice. Only four of the studies were conducted outside of the United States.
Methodology of studies
Four of the studies involved a randomized controlled trial study design (Table 5 of online ESM). Most of the investigations were of a quasi-experimental design. The remaining three studies utilized a retrospective category of methodology.
Inclusion and exclusion criteria were specified in approximately half of the studies. A feedback tool was utilized in 14 of the studies. Three of the studies had longer-term follow-up, ranging from 4 months to 5 years.
The sample size of participants ranged from 7 to 96, with roughly half of the studies involving participant sizes less than 30, ten in the 31–49 grouping and two greater than 50. The number of participants was not stated in one study.
Discussion
Patient outcomes feedback is currently widely used at the practising health professional level as a strategy to improve practice performance both on its own and as a component of multifaceted quality improvement interventions [46]. This type of feedback has been shown to successfully improve physician clinical performance and compliance with quality measures [47], especially when the feedback is provided systematically by a reliable source over time [13]. Less is known about the effect of feedback on patient outcomes to medical learners, an important topic for medical education. In the Kirkpatrick Model [48], designed for evaluating the effectiveness of training across four levels, these outcomes represent the highest level (results).
The findings from this descriptive analysis add to the current medical literature by describing what is known about the association of performance feedback to medical learners with characteristics of patient care. Encouragingly, despite deficiencies in rigor and scope, most studies showed a positive relationship between feedback to medical learners and improvement in patient-related outcomes.
Patient care is broadly defined as “the services rendered by members of the health professions for the benefit of a patient [49]”. Most of the identified studies focused on patient care in the areas of healthcare management and practice performance such as adherence to practice guidelines and immunization rates, improving documentation and procedural and physical examination skills. Although we presume indirect outcomes such as improved documentation and procedural skills translate to improved quality of care, more specific studies would be required to precisely document this assumed benefit. There was very little data on the effectiveness of feedback on more direct patient outcomes. This speaks to the inherent difficulty of designing studies that directly relate feedback to measurable patient care outcomes, involve parameters that are generally agreed to be of prime importance, and provide usable data that aligns with learner competency milestones.
The data obtained show that there are few studies devoted to this topic and most were conducted in the short term with small numbers. Very few of the included studies assessed for bias (unforeseen confounding variables), used assessment tools that had undergone validation studies or involved rigorous study designs. This is perhaps secondary to the belief that medical education research may lend itself more to descriptive and clarification studies than the gold standard of randomized controlled trials (RCTs) [17]. Sullivan, for example, suggests that attention to elements such as the use of a comparison group, adequate numbers, recognizing bias and a thoughtful discussion of limitations may be more important than a focus on a RCT study design in medical education research [50].
The identified studies were primarily conducted at a single university teaching hospital in the US and with a single medical discipline, most commonly internal medicine. It would be necessary to conduct studies in more diverse settings, involving multiple disciplines at multiple institutions to make the findings more generalizable. At the very least, involving a broader spectrum of disciplines in these types of studies would add to the medical feedback literature and inform useful methods for helping learners to meet discipline specific competencies and milestones in patient care.
Research that gives attention to the feedback method and delivery, including the contextual factors surrounding the feedback process [51], may help to facilitate the effectiveness of the patient outcomes information. Aligning outcomes studies with the increasing focus on learner characteristics and perceptions that promote feedback acceptance, as well as the alliance between the teacher and the learner [52], will add to the current medical learner feedback literature. A focus of establishment of standards in measuring patient outcomes will allow providers to collect and share data on outcomes more efficiently, facilitate comparisons that will accelerate improved patient care [53] and guide appropriate patient outcomes goals for learners. Positioning learners as key drivers in their own education and encouraging them to solicit and generate their own patient outcomes feedback may facilitate advancement in this area as well [54].
Limitations
Overall, the small number of studies included in this analysis and their lack of rigorous study design hamper our ability to draw strong conclusions about the data. It may be difficult to separate the effect of the student impact on patient outcomes from the effect of the team or supervising physician. There is a theoretical risk of publication bias due to the likelihood of studies with positive outcomes being more likely to be submitted and accepted for publication than studies showing no change or a negative outcome. Another limitation is the lack of generalizability due to the preponderance of single discipline, single institution studies.
Conclusion
Published studies focusing on the association of medical learner feedback with characteristics of patient care are sparse. Most involve a single discipline at a single institution and are of a non-rigorous design. Measurements of improved patient outcomes are restricted to changes in management, procedures and documentation. Well-designed studies that more directly link learner feedback to patient outcomes may help to support the use of feedback in teaching clinical outcomes improvement in alignment with competency-based milestones.
Conflict of interest
V. Hayes, R. Bing-You, K. Varaklis, R. Trowbridge, H. Kemp and D. McKelvy declare that they have no competing interests.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.