Skip to main content
Top
Gepubliceerd in: Perspectives on Medical Education 4/2022

Open Access 08-04-2022 | Original Article

Bridging medical education goals and health system outcomes: An instrumental case study of pre-clerkship students’ improvement projects

Auteurs: Bridget C. O’Brien, Josué Zapata, Anna Chang, Edgar Pierluissi

Gepubliceerd in: Perspectives on Medical Education | Uitgave 4/2022

share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail
insite
ZOEKEN

Abstract

Introduction

Many medical schools engage students in health system improvement (HSI) efforts. Evaluation of these efforts often focuses on students’ learning outcomes and rarely considers the impact on health systems, despite the significant commitment health systems make to these efforts. Our study identified and evaluated system-level outcomes of pre-clerkship medical students’ engagement in HSI efforts.

Methods

We used an instrumental case study approach to examine the effects of pre-clerkship medical students’ engagement in HSI projects as part of a 15-month experiential curriculum. We extracted data from 53 project summaries and posters completed during the 2017–18 academic year and follow-up survey data collected in May 2019 from physician coaches and health system professionals who mentored students, contributed to these projects, and worked in the clinical microsystems where the projects occurred.

Results

We identified three categories and ten indicators of health system outcomes relevant to medical student engagement in HSI. Using these indicators, our evaluation found multiple benefits to the microsystems in which projects occurred. These included achievement of project aims, perceived immediate and sustained project impact on the health system, and development and implementation of projects with aims that aligned with national and health system priorities.

Conclusion

Evaluation of HSI curricula needs to include effects on health systems so that program design can optimize the experience for all involved. Our study offers a framework others can use to evaluate system-level effects of project-based HSI curricula and shows several ways in which students’ engagement can add value to health systems.

Introduction

Healthcare systems face many challenges that medical schools and graduate medical education programs must help address [13]. The inclusion of systems-based practice as one of six Accreditation Council for Graduate Medical Education (ACGME) core competencies for physicians in 1999 signalled a commitment to make medical education part of the solution to gaps in healthcare quality, disparities, and value [4, 5]. Educators have identified similar competencies for medical students [6, 7] and proposed health systems science as an overarching framework that covers “the methods and principles for improving quality, outcomes, and costs of healthcare delivery for patients and populations within medical care systems” [8, p. 123–124]. While these efforts represent important strides toward defining desired learning outcomes that can be used to evaluate curricular success from an educational standpoint, the indicators of success from a health systems standpoint are less clear. If medical education aspires to be part of the solution, greater clarity is needed about ways in which educational initiatives may positively affect health systems. In this paper, we explore the system-level effects of an effort to engage medical students in health systems improvement prior to clerkships.
In 2013, the American Medical Association launched “Accelerating Change in Medical Education,” an initiative that challenged medical schools to design curricula that prepare physicians capable of navigating and improving health systems to benefit patients and communities [9]. The approaches to teaching health systems science vary among medical schools [10], but a common guiding principle is to develop students’ competency in health systems science through authentic experiences while helping health systems achieve quality and value goals [1113]. Health systems improvement (HSI), or quality improvement, involves “identifying, analyzing, or implementing changes in … the healthcare system to improve the performance of any component of the healthcare system” [8, p. 125]. Many medical schools have partnered with health systems to engage learners in improvement efforts [9, 10, 14, 15]. These partnerships are compelling from an educational standpoint as they provide opportunities to develop key HSI competencies (e.g., quality improvement principles; data and measurement; innovation and scholarship) [16] through workplace-based experiential learning, which is widely recognized as an effective means of education [1719].
The New World Kirkpatrick Model [20] defines four levels of outcomes in medical education: Level 1) reaction—learner satisfaction and engagement; Level 2) learning—modification of attitudes, knowledge and skills; Level 3) behaviour—application of learning to real-world practices; and Level 4) results—changes in patient care, new or improved organizational practices or performance, societal change [2022]. In the evaluation of undergraduate medical education curricula, level 4 is often omitted because it is resource-intensive to measure and challenging to link directly to medical students’ actions [2123]. Yet, the activity of students as part of an interdisciplinary HSI team provides an opportunity to include measurement of structure, process, and patient outcomes as well as perspectives of key health system stakeholders [24]. HSI often generates change on longer timeframes, making intermediary outcomes such as stakeholders’ perceptions critical to sustaining the partnership between medical education and health systems that invite student involvement in systems improvement. Without stakeholder buy-in, the commitment of human and other resources from partnering health systems is at risk. These types of level 4 outcomes, from the systems perspective, have rarely been explored. Our study aims to define and evaluate system-level outcomes of students’ engagement in HSI (stakeholder perspectives, structure, process, patient outcomes) in the hope of generating a framework that can guide the evaluation of experiential HSI curricula.

Methods

Context

All medical students in their first 15 months at the University of California San Francisco (UCSF) participate in an experiential HSI curriculum as part of the Clinical Microsystem Clerkship (CMC) [25, 26]. The clinical microsystems are housed within one of three health systems (academic, safety net, and veterans’ affairs). Student teams work with a physician coach and staff from the clinical microsystem on a project that addresses a health system problem that is identified prior to students’ arrival. These problems and projects are selected through criteria developed by CMC curriculum directors with input from health system leaders. Key criteria include alignment with national/institutional goals, involvement of interprofessional team members, interaction with patients, consideration of equity and disparities, feasibility to implement over 15 months, and data available to inform the project. This curriculum allows students to learn health systems science through real-world application of concepts [17, 18] while advancing health system goals [11].
Within the HSI curriculum, students are expected to apply Lean A3 improvement methodology [27] to a current health system problem. Students work with health professionals in the microsystem to articulate the problem their project will address, define improvement goals, perform a gap analysis, implement interventions, evaluate outcomes, and reflect on the learning process. The curriculum includes lectures, workshops, and projects. Each group receives guidance from a physician coach and health system QI team members. Student groups submit four interim project summaries and present their completed work as posters to education deans and health system leaders. Our study was reviewed and granted exempt status by our institution’s committee for human research (File number: #19-27272).

Design

We selected an instrumental case study approach [28], which is the study of a specific example to provide insight into a more general issue or phenomenon. In our study, the case is one academic year of the experiential, project-based portion of the UCSF HSI curriculum and the issue is how to incorporate health system-level outcomes into the evaluation of experiential HSI curricula. We drew on the New World Kirkpatrick Model [20], focusing on level 4 (results) outcomes that could bridge educational and health systems goals of students’ engagement in HSI efforts. Through a combination of literature review, conversations with local health system leaders about essential criteria for students’ HSI projects from the health system perspective, and consideration of available data, we grouped systems-level outcomes of interest into three categories: 1. Project goal completion, 2. Effects on the microsystem, and 3. Project alignment with health system (see Resource 1 of the Electronic Supplementary Material (ESM)).

Data sources and data collection

We included all projects completed by medical students during the second year the CMC existed at our institution (August 2017 through November 2018). We chose this year to capture effects of improvements made to stabilize the curriculum after the launch year. Also, at the point of data collection, several months had passed since students had completed the CMC, which allowed us to find out about downstream effects of their efforts.
We analyzed data collected from three sources (see ESM resource 2): Project summaries completed by student teams using an A3 format to document the steps of their project [29], Project posters presented at health system quality improvement (QI) forums, and a Survey of physician coaches and health system QI project leader (QI leads) (ESM resource 3). We chose to survey key health system stakeholders rather than institutional leadership because of their direct experience with the HSI curriculum and knowledge of how students’ efforts affected the microsystem [25, 30]. We used existing literature [24, 31] and information extracted from project summaries and posters to develop the survey. Questions asked respondents to describe changes in the microsystem due to the project, to rate the projects’ effect on the microsystem, to state whether effects were sustained, and to identify reasons for these ratings. We piloted the questionnaire by conducting cognitive interviews with two physician coaches and modified questions to enhance clarity based on the feedback received [32]. The final version of the questionnaire was uploaded to Qualtrics, a web-based survey platform, and one team member (JZ) sent personal invitations to each faculty coach and QI lead to participate.
We created a data extraction form to collect descriptive information about project characteristics, aims, interventions, data/metrics, achievement of aims, and barriers/facilitators to project success from project summaries and posters (ESM resource 4). We coded project aims according to whether the primary aim was to change structural, process, or patient outcome measures (see ESM resource 1 for a description). All authors independently tested the extraction form and as a group agreed to a final version. After finalizing the form, we worked in pairs to extract data from all 53 projects. JZ coded all 53 project summaries and posters, the other three investigators coded one-third. Coding discrepancies were reconciled to consensus. Data were recorded in Microsoft excel.

Data analysis

We calculated descriptive statistics for all categorical and numeric data. Since some projects received multiple survey responses from coaches and QI leads, we aggregated data and used the mean response per project. We coded free text responses to identify categories and themes [33].

Results

Second year medical students (n = 152) completed 53 projects that spanned a broad range of clinical microsystems across three affiliated health systems, five types of clinical settings, and 16 medical sub-specialties. Tab. 1 describes the key characteristics of the projects, Box 1 shows a sample project; a description of two further sample projects can be found in the ESM Box and ESM resource 5 lists all the projects. Twenty-six coaches (96%) and 13 QI leads (68%) completed follow-up surveys for 49 of the projects (92%) (ESM resource 3). We organized our findings into three categories of system-level outcomes with multiple indicators for each category. The first category represents immediate outcomes (accomplishment of project goals), the second captures effects after the project ended, and the third describes project alignment with health system priorities and processes.
Table 1
Key features of medical students’ clinical microsystem clerkship health systems improvement projects, 2017–18
Key features
No. projects
(% of 53)
Health system
 Academic
22 (42)
 Safety net
19 (36)
 Veterans affairs
12 (23)
Clinical microsystem type
 Adult medicine
48 (91)
 Pediatric medicine
 5 (9)
 Ambulatory care
27 (51)
 Acute care
18 (34)
 Surgical/perioperative care
 7 (13)
 Other
 1 (2)
Project aims to improve … (IOM priorities34 projects may address multiple) (Indicator 3.1)
 Effectiveness
41 (77)
 Safety
18 (34)
 Patient-centeredness
10 (19)
 Efficiency
 9 (17)
 Equity
 8 (15)
 Timeliness
 5 (9)
Type of measure (Indicator 2.3)
 Structural change
 1 (2)
 Process change
36 (68)
 Patient outcome
16 (30)
Project achieved at least 1 aim (based on data provided in project summary report or poster) (Indicators 1.1 and 2.3)
Yes
28 (53)
– Structural changes
 1
– Process changes
19
– Patient outcomes
 8
No
24 (45)
– Process changes
16
– Patient outcomes
 8
Could not determine
 1 (2)
– Process changes
 1
Box 1 Description of a sample project
Improving hypertension control in black patients at an academic primary care clinic
This project focused on reducing a healthcare disparity in hypertension control in a primary care clinic. At baseline, the rate of uncontrolled hypertension among Black patients in this primary care clinic was significantly higher (35%) than that of patients of all other races (26%). The student team set a goal of closing this gap and reducing the rate of uncontrolled hypertension among Black patients to that of patients of all other races within six months. To explore potential reasons for this disparity, students performed a literature review, interviewed nursing staff, the clinic practice manager, and patients. They learned that the underlying causes varied from patient to patient. They designed an intervention in which a student on the team called each Black patient with uncontrolled hypertension to discuss the patient’s understanding of hypertension, hear their concerns about their ability to manage this condition, and schedule follow-up clinic appointments, if appropriate. Over the course of the project, the rate of uncontrolled hypertension among Black patients decreased from 35% to 30%, short of the team’s stated goal of 26%. However, the project increased utilization of blood pressure appointments with nurses which were often more convenient for patients. It also initiated a change across the health system to the electronic medical record so that all office visit blood pressure measurements are utilized in determining hypertension control status, not just those from primary care. At the end of the students’ curricular time, the effort was taken over by clinic staff and the scope increased to encompass all primary care clinics within the academic medical center. The clinic QI lead reported that this project improved their microsystem, commenting that “embedded telephone outreach and follow-up workflow is now being developed, in large part due to the enthusiasm and results from the students’ intervention.” Additionally, the students achieved learning goals in the areas of quality improvement and health disparities.

Category 1: Project goals accomplished by end of the curriculum

Indicator 1.1 Achievement of project aims: More than half (53%) of the project summaries and posters indicated achievement of at least one aim (Tab. 1).
Indicator 1.2 Achievement of educational goals: Survey data indicated that most physician coaches and QI leads (86%) agreed that students achieved the primary educational goal of applying HSI principles. Respondents commented on students’ appreciation of the complexities of quality improvement, valuing patients and interprofessional colleagues as key stakeholders in the improvement process, applying the steps of Lean methodology through a hands-on experience, conducting outcome measurement from Plan-Do-Study-Act (PDSA) cycles, and presentation skills. These findings suggest that members of the health system felt that students learned to apply HSI knowledge and skills, a potentially important health system goal since many students will go on to train and work in these settings.

Category 2: Effects of the HSI project on the system

Indicator 2.1 Perceived effects at the end of the curriculum (15 months): Based on coach and QI lead survey responses, many projects (67%) had a moderate or substantial impact on the microsystem at the time students completed their involvement with the project (Tab. 2)—even if project aims were not achieved. For projects perceived as having minimal or no effect on the microsystem, reasons included structural or bureaucratic barriers (e.g. difficulty getting intervention approved), insufficient time for students to complete the project (e.g. poor project management, unrealistic scope, or schedule conflicts that limited participation in key activities), and project completed or adopted by another group separate from students’ efforts.
Table 2
Perceived impact of medical students’ clinical microsystem clerkship health systems improvement projects from 2017–18 based on follow-up survey responses from coaches and QI leads (collected in May 2019)
Survey responses and sample explanations from free response
No. projects
(% of 49a)
Indicator 2.1 Perceived impact at project end (15 months)
Impact of project on microsystem at time students left (approx. Nov 2018)
 None (e.g., did not meet proposed goal)
 3 (6)
 Minimal (e.g., project part of larger effort, hard to connect outcomes to students’ efforts, persistent structural barriers)
13 (27)
 Moderate (e.g., students sparked interest that increased outreach efforts, pilot phase)
23 (47)
 Substantial (e.g., improvement in valued metrics such as documentation, screening, illuminated a problem that was then addressed, provided a framework or foundation for future efforts)
10 (20)
Indicator 2.2 Perceived lasting impact of project (7-month post-curriculum)
Did the project have impact after the students left the microsystem? (approx. May 2019)
Yes because …
37/49 (76)
– Project (including all, some piece of it, or next phase) was taken on by health system staff
26/37 (70)
– Project has been ‘hardwired’ into the microsystem, requires minimal effort to sustain
16/37 (43)
– Project (including all, some piece of it, or next phase) was taken on by a new cohort of students
 3/37 (8)
– Other (e.g., educational efforts help sustain change; culture shift among key health professionals in ways that support the intervention; identified a need for outside consultants who then had a huge impact)
 5/37 (14)
No because …
12/49 (25)
– When the students left there was no one to do the work
 6/12 (50)
– Project was no longer relevant (circumstances/priorities changed)
 3/12 (25)
– Project identified other areas that needed to be prioritized
 2/12 (17)
– Other (e.g., lack of buy in from key health professionals)
 5/12 (42)
a Survey responses received from coaches and/or QI leads of 49 out of 53 projects. Multiple responses for the same project were aggregated.
Indicator 2.2 Perceived effects post-curriculum (7 months post-curriculum): Based on coach and QI lead survey responses, many projects (76%) had a sustained impact on the microsystem seven months later (Tab. 2). One respondent described how student engagement could inspire subsequent efforts, “Student work sparked interest in focusing specifically on hypertension control for African Americans in the broader primary care organization. Outreach efforts were expanded.” Coaches and QI leads attributed sustainability to health system staff continuing the improvement work and/or incorporation of interventions into usual processes. The most commonly cited reasons projects were not sustained included a shift in health system priorities, and lack of resources to continue the work without students, as one respondent wrote, “There was no one to push to work forward after the students left and the project fell by the wayside.”
Indicator 2.3 Structures, processes, and patient outcomes: Projects focused on changing structures (1 project), processes (36 projects), or patient outcomes (16 projects). One project made a structural change by producing a handout about delirium for family members of patients in the intensive care unit. Of the 36 projects focused on process changes, 19 (53%) achieved their primary aim. For example, the depression screening rate in a primary care clinic increased from 58% to 63%, the median time from door to thrombolytic infusion in patients with stroke decreased from 38 to 30 min, and the proportion of psychiatric inpatients attending a first outpatient mental appointment increased from 55% to 64%. Of the 16 projects focused on changing a patient outcome measure, 8 (50%) achieved their primary aim. For example, the rate of in-hospital injury from falls decreased from 0.58 to 0.52 per 1000 patients and influenza vaccination rates increased from 37 to 58% in a primary care clinic.
Indicator 2.4 Balance of costs and benefits: While our data did not allow a true cost-benefit analysis, information about barriers and facilitators to project success gave us insight into some required resources or investments from the system to support the HSI projects. These included staff time to meet with students, participate in the design and pilot implementation of interventions, as well as helping students navigate the microsystem (e.g., accessing data and connecting them with key stakeholders and local champions). Additional time for coaches and QI leads to assist with project management was also important.
The benefits described by coaches and QI leads largely focused on the educational value to students such as learning and experiencing “the steps of QI work,” “how to work with a multidisciplinary team,” and “the crucial role of getting buy-in from all stakeholders.” While some coaches and QI leads highlighted small but valuable improvements from students efforts, such as raising awareness of gaps in care, introducing processes that helped streamline and standardize care, figuring out why a proposed intervention might not work and “improving the quality of the (patient satisfaction) tool and testing workflow for its administration”, others felt it was difficult or even unrealistic “for students to truly improve a system with the amount of time they have.” Nonetheless, many saw the value of the system long-term, noting “if they get an exposure to Lean and design thinking and QI methods, that is itself a win.”

Category 3: Project alignment with health system priorities and processes

Indicator 3.1 Alignment with health system or microsystem goals: From our analysis of all three data sources, we found many signs of project alignment with health system priorities. We used Institute of Medicine (IOM) priorities [34] as all three health systems embraced these. All projects addressed one or more IOM priority area; improving the effectiveness of care was most common (Tab. 1). Most projects also addressed a national and/or local priority (77%). Nearly half referenced a published benchmark (45%). Addressing a healthcare disparity was an explicit goal in 15% of projects.
Indicator 3.2 Involvement of key stakeholders: Analysis of project reports and posters showed that all student teams engaged clinicians from multiple health professions in one or more core improvement step. Professions included nursing, pharmacy, physical therapy, psychology, social work, and data science as well as administrative and research personnel. Twenty-three projects (43%) involved direct interactions between students and patients or families to understand the gap or to design, implement, or measure the intervention. Projects that did not involve direct interaction with patients focused on interventions for clinicians (e.g., note templates or clinician training). These findings suggest success in engaging key stakeholders from clinical microsystems in students’ HSI projects.
Indicator 3.3 Use of health system improvement tools and interventions: We found that projects used tools familiar to the health systems in which the projects occurred. Most projects (87%) used fishbone diagrams to represent their gap analysis. Others used process maps, Possible-Implement-Challenge-Kill (PICK) charts, and five why’s [35, 36]. This finding suggests alignment between the language, concepts and tools students learned in the formal medical school curriculum and the language, concepts and tools used in the clinical microsystems. Projects often included more than one intervention (74%) reflecting sensitivity to the need for multi-pronged approaches. Almost all projects (89%) attempted to provide in-process support (e.g. team huddles, or electronic health record templates, alerts, or checklists) that could enhance existing processes in the microsystem. Other common interventions included training and education for healthcare professionals, staff, patients, and/or family members (72%) and audit and feedback (26%).
Indicator 3.4 Reasonable scope for implementation: Most projects (79%) fully implemented at least one intended intervention within the timeframe of the curriculum and 19% partially implemented at least one intervention. All but one project completed at least one PDSA cycle. The average duration of the intervention period was five months. These findings suggest that projects could be designed and managed to support completion of specific goals and activities within the expected 15 month period.

Discussion

As medical schools engage students in health systems improvement, the effects of such curricula need to be defined in ways that address the goals of stakeholders in both the educational and health systems enterprise. The New World Kirkpatrick model [20] is widely used to evaluate learning outcomes at the individual level, with less attention to system-level outcomes. We identified and used three general outcome categories and ten specific indicators to evaluate the clinical microsystem-level effects of a project-based HSI curriculum (Kirkpatrick level 4). In our study, 30% of projects addressed patient outcomes and nearly half of these achieved their aims. We found that two-thirds of projects were perceived to have moderate or substantial impact on the microsystem, often with effects that extended beyond students’ time in the curriculum. While these effects did not always correspond to achievement of projects’ specific aims or targeted metrics, other types of benefits accrued such as new insight into a problem, staff education or heightened awareness of an issue, or recognition of reasons why a proposed intervention would not work. While projects required resources, they generally aligned with health system priorities and processes. Based on our findings, we discuss implications of our work for future efforts to evaluate project-based HSI curricula.
Our study adds to recent literature that seeks to understand how students can add value to both individuals and systems for healthcare organizations that offer clinical placements to trainees [3739]. Two reviews [37, 38] focused on individual-level effects on health system educators and suggested that the benefits educators experience when working with students (e.g., enjoyment of teaching, opportunity to gain new knowledge and improve skills) may outweigh costs (e.g., increased workload, decreased productivity). We did not explicitly collect data about perceived benefits and burdens among staff who worked with students in the microsystem, though some project summaries and survey comments on barriers, facilitators and perceived impact addressed this. In retrospect we see this information as an important component of value to health systems and recommend including it in cost-benefit indicators.
Few studies have explored organizational level effects of student engagement in systems improvement [39]. Based on interviews with placement educators, Kemp and colleagues [39] found that students were perceived as adding to workforce capacity by moving valued projects forward, creating ‘physical deliverables’ such as resources and videos, creating new partnerships and launching new projects. Challenges included costs such as time invested in planning for and supervising students and working with students who lacked necessary professional skills to complete tasks independently. The authors found that over time, the organizational benefits increased and costs decreased with more trust and stronger partnership between the organizations and the university. These findings underscore the importance of including system-level indicators as part of routine evaluation so effects can be monitored over time and experiences that could diminish trust between medical schools and clinical microsystem partners can be addressed.
Our findings also illuminate other challenges encountered by students, coaches and staff. Some of these challenges such as project scope, access to data, and schedule conflicts may be anticipated and provide useful guidance for future project planning. Other challenges such shifting organizational priorities, loss of key personnel, and delays in implementing changes to the electronic medical record may be harder to anticipate.
Prior work has suggested value-added roles for students [11], but to our knowledge has not followed up to examine the perceived or actual value of students’ contributions to clinical microsystems. One reason may be that value is often conceptualized in monetary terms, yet many of the valued outcomes are difficult to monetize—particularly in a way that is feasible for routine program evaluation. Our study offers a way to conceptualize value in broader terms based on academic products (project summaries, posters) and stakeholder perceptions. Subsequent work may benefit from including a broader array of stakeholders such as additional staff, patients, and administrators in the health system whose work or experience have been affected by the students’ efforts.

Limitations

We derived categories and indicators for microsystem-level outcomes based in part on the HSI curriculum and data available at our institution, which may limit the transferability of indicators to other contexts. Nonetheless, the three general categories can be used as a starting point for identifying indicators specific to different settings. We collected data from sources that may emphasize positive outcomes and under-report critiques, challenges, or unsuccessful aspects of the experience. While survey respondents may have been reluctant to say the projects did not have an impact, the fact that some admitted challenges and explained reasons gives us some encouragement about the credibility of the data. Additional sources of information could be sought beyond the coaches and QI leads, perhaps from external reviewers who were less directly involved in projects, to gauge impact.

Conclusion

As project-based health systems improvement curricula become more common in medical student education, comprehensive evaluation plans that include realistic, meaningful indicators of system-level outcomes along with student learning outcomes will provide critical information to sustain strong partnerships between medical schools and health systems. Our study demonstrates the value of including the perspectives of health system physicians and clinical staff who observe the effects of students’ contributions first-hand, as well as their impact on the clinical microsystem after the curriculum ends. Through such engagement of key stakeholders, these curricula can be further adapted to drive outcomes that matter not just to students and educators, but also to health systems leaders and to the patients that these systems serve.

Acknowledgements

The authors wish to thank Patricia Runquist for assistance with data collection and Drs. Patricia O’Sullivan and Arianne Teherani for helpful feedback on the manuscript.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://​creativecommons.​org/​licenses/​by/​4.​0/​.
Literatuur
1.
go back to reference Berwick DM, Finkelstein JA. Preparing medical students for the continual improvement of health and health care: Abraham Flexner and the new “public interest”. Acad Med. 2010;85:S56–S65.CrossRef Berwick DM, Finkelstein JA. Preparing medical students for the continual improvement of health and health care: Abraham Flexner and the new “public interest”. Acad Med. 2010;85:S56–S65.CrossRef
2.
go back to reference Lucey CR. Medical education: Part of the problem and part of the solution. JAMA Intern Med. 2013;173:1639.CrossRef Lucey CR. Medical education: Part of the problem and part of the solution. JAMA Intern Med. 2013;173:1639.CrossRef
3.
go back to reference Sklar DP, Hemmer PA, Durning SJ. Medical education and health care delivery: A call to better align goals and purposes. Acad Med. 2018;93:384–90.CrossRef Sklar DP, Hemmer PA, Durning SJ. Medical education and health care delivery: A call to better align goals and purposes. Acad Med. 2018;93:384–90.CrossRef
4.
go back to reference Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S. General competencies and accreditation in graduate medical education. Health Aff. 2002;21:103–11.CrossRef Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S. General competencies and accreditation in graduate medical education. Health Aff. 2002;21:103–11.CrossRef
5.
go back to reference Johnson JK, Miller SH, Horowitz SD. Systems-based practice: improving the safety and quality of patient care by recognizing and improving the systems in which we work. In: Henriksen K, Battles JB, Keyes MA, Grady ML, editors. Culture and redesign. Advances in patient safety: new directions and alternative approaches, Vol. 2. Rockville: Agency for Healthcare Research and Quality; 2008. Johnson JK, Miller SH, Horowitz SD. Systems-based practice: improving the safety and quality of patient care by recognizing and improving the systems in which we work. In: Henriksen K, Battles JB, Keyes MA, Grady ML, editors. Culture and redesign. Advances in patient safety: new directions and alternative approaches, Vol. 2. Rockville: Agency for Healthcare Research and Quality; 2008.
6.
go back to reference Association of American Medical Colleges. Contemporary issues in medicine: Quality of care. Medical school objectives project, report II. Washington: AAMC; 2001. Association of American Medical Colleges. Contemporary issues in medicine: Quality of care. Medical school objectives project, report II. Washington: AAMC; 2001.
7.
go back to reference Association of American Medical Colleges. Core entrustable professional activities for entering residency: curriculum developers’ guide. Washington: Association of American Medical Colleges; 2014. Association of American Medical Colleges. Core entrustable professional activities for entering residency: curriculum developers’ guide. Washington: Association of American Medical Colleges; 2014.
8.
go back to reference Gonzalo JD, Dekhtyar M, Starr SR, et al. Health systems science curricula in undergraduate medical education: Identifying and defining a potential curricular framework. Acad Med. 2017;92:123–31.CrossRef Gonzalo JD, Dekhtyar M, Starr SR, et al. Health systems science curricula in undergraduate medical education: Identifying and defining a potential curricular framework. Acad Med. 2017;92:123–31.CrossRef
9.
go back to reference Skochelak SE, Stack SJ. Creating the medical schools of the future. Acad Med. 2017;92:16–9.CrossRef Skochelak SE, Stack SJ. Creating the medical schools of the future. Acad Med. 2017;92:16–9.CrossRef
11.
go back to reference Gonzalo JD, Graaf D, Johannes B, Blatt B, Wolpaw DR. Adding value to the health care system: Identifying value-added systems roles for medical students. Am J Med Qual. 2017;32:261–70.CrossRef Gonzalo JD, Graaf D, Johannes B, Blatt B, Wolpaw DR. Adding value to the health care system: Identifying value-added systems roles for medical students. Am J Med Qual. 2017;32:261–70.CrossRef
12.
go back to reference O’Brien BC, Bachhuber MR, Teherani A, Iker TM, Batt J, O’Sullivan PS. Systems-oriented workplace learning experiences for early learners: Three models. Acad Med. 2017;92:684–93.CrossRef O’Brien BC, Bachhuber MR, Teherani A, Iker TM, Batt J, O’Sullivan PS. Systems-oriented workplace learning experiences for early learners: Three models. Acad Med. 2017;92:684–93.CrossRef
13.
go back to reference Gonzalo JD, Baxley E, Borkan J, et al. Priority areas and potential solutions for successful integration and sustainment of health systems science in undergraduate medical education. Acad Med. 2017;92:63–9.CrossRef Gonzalo JD, Baxley E, Borkan J, et al. Priority areas and potential solutions for successful integration and sustainment of health systems science in undergraduate medical education. Acad Med. 2017;92:63–9.CrossRef
14.
go back to reference Elliott VS, Jackson J, Santen SA, Richardson J, Heckman K, Hammoud MM. Triple challenge: How medical students may solve 3 long-standing problems bedeviling health systems and medical education. Acad Med. 2021;96:1643–9.CrossRef Elliott VS, Jackson J, Santen SA, Richardson J, Heckman K, Hammoud MM. Triple challenge: How medical students may solve 3 long-standing problems bedeviling health systems and medical education. Acad Med. 2021;96:1643–9.CrossRef
15.
go back to reference Brown A, Lafreniere K, Freedman D, et al. A realist synthesis of quality improvement curricula in undergraduate and postgraduate medical education: what works, for whom, and in what contexts? BMJ Qual Saf. 2021;30:337–52.CrossRef Brown A, Lafreniere K, Freedman D, et al. A realist synthesis of quality improvement curricula in undergraduate and postgraduate medical education: what works, for whom, and in what contexts? BMJ Qual Saf. 2021;30:337–52.CrossRef
16.
go back to reference Gonzalo JD, Chang A, Dekhtyar M, Starr SR, Holmboe E, Wolpaw DR. Health systems science in medical education: Unifying the components to catalyze transformation. Acad Med. 2020;95:1362–72.CrossRef Gonzalo JD, Chang A, Dekhtyar M, Starr SR, Holmboe E, Wolpaw DR. Health systems science in medical education: Unifying the components to catalyze transformation. Acad Med. 2020;95:1362–72.CrossRef
17.
go back to reference Yardley S. Sense made common: how to add value to early experience. Clin Teach. 2014;11:5–9.CrossRef Yardley S. Sense made common: how to add value to early experience. Clin Teach. 2014;11:5–9.CrossRef
18.
go back to reference Yardley S, Teunissen PW, Dornan T. Experiential learning: transforming theory into practice. Med Teach. 2012;34:161–4.CrossRef Yardley S, Teunissen PW, Dornan T. Experiential learning: transforming theory into practice. Med Teach. 2012;34:161–4.CrossRef
19.
go back to reference Goldman J, Kuper A, Wong BM. How theory can inform our understanding of experiential learning in quality improvement education. Acad Med. 2018;93:1784–90.CrossRef Goldman J, Kuper A, Wong BM. How theory can inform our understanding of experiential learning in quality improvement education. Acad Med. 2018;93:1784–90.CrossRef
20.
go back to reference Kirkpatrick JD, Kirkpatrick WK. Kirkpatrick’s four levels of training evaluation. Alexandria: ATD Press; 2016. Kirkpatrick JD, Kirkpatrick WK. Kirkpatrick’s four levels of training evaluation. Alexandria: ATD Press; 2016.
21.
go back to reference Onyura B, Baker L, Cameron B, Friesen F, Leslie K. Evidence for curricular and instructional design approaches in undergraduate medical education: An umbrella review. Med Teach. 2016;38:150–61.CrossRef Onyura B, Baker L, Cameron B, Friesen F, Leslie K. Evidence for curricular and instructional design approaches in undergraduate medical education: An umbrella review. Med Teach. 2016;38:150–61.CrossRef
22.
go back to reference Moreau KA. Has the new Kirkpatrick generation built a better hammer for our evaluation toolbox? Med Teach. 2017;39:999–1001.CrossRef Moreau KA. Has the new Kirkpatrick generation built a better hammer for our evaluation toolbox? Med Teach. 2017;39:999–1001.CrossRef
23.
go back to reference Gylys R, Rosenwohl-Mack S, Pierluissi E, Hoffman A. Assessing contributions of value-added medical student roles. Med Teach. 2021;43:232–3.CrossRef Gylys R, Rosenwohl-Mack S, Pierluissi E, Hoffman A. Assessing contributions of value-added medical student roles. Med Teach. 2021;43:232–3.CrossRef
24.
go back to reference Donabedian A. The quality of care: How can it be assessed? JAMA. 1988;260:1743–8.CrossRef Donabedian A. The quality of care: How can it be assessed? JAMA. 1988;260:1743–8.CrossRef
25.
go back to reference Harbell MW, Li D, Boscardin C, Pierluissi E, Hauer KE. Teaching systems improvement to early medical students: Strategies and lessons learned. Acad Med. 2020;95:136–44.CrossRef Harbell MW, Li D, Boscardin C, Pierluissi E, Hauer KE. Teaching systems improvement to early medical students: Strategies and lessons learned. Acad Med. 2020;95:136–44.CrossRef
26.
go back to reference Gonzalo JD, Chang A, Wolpaw DR. New educator roles for health systems science: Implications of new physician competencies for U.S. medical school faculty. Acad Med. 2019;94:501–6.CrossRef Gonzalo JD, Chang A, Wolpaw DR. New educator roles for health systems science: Implications of new physician competencies for U.S. medical school faculty. Acad Med. 2019;94:501–6.CrossRef
27.
go back to reference Mazzocato P, Savage C, Brommels M, Aronsson H, Thor J. Lean thinking in healthcare: a realist review of the literature. Qual Saf Health Care. 2010;19:376–82. Mazzocato P, Savage C, Brommels M, Aronsson H, Thor J. Lean thinking in healthcare: a realist review of the literature. Qual Saf Health Care. 2010;19:376–82.
28.
go back to reference Yazan B. Three approaches to case study methods in education: Yin, Merriam, and Stake. Qual Rep. 2015;20:134–52. Yazan B. Three approaches to case study methods in education: Yin, Merriam, and Stake. Qual Rep. 2015;20:134–52.
29.
go back to reference Shook J. Managing to learn: Using the A3 management process to solve problems, gain agreement, mentor, and lead. Cambridge: Lean Enterprise Institute; 2008. Shook J. Managing to learn: Using the A3 management process to solve problems, gain agreement, mentor, and lead. Cambridge: Lean Enterprise Institute; 2008.
30.
go back to reference Likosky DS. Clinical microsystems: a critical framework for crossing the quality chasm. J Extra Corpor Technol. 2014;46:33–7. Likosky DS. Clinical microsystems: a critical framework for crossing the quality chasm. J Extra Corpor Technol. 2014;46:33–7.
31.
go back to reference Reed JE, Kaplan HC, Ismail SA. A new typology for understanding context: qualitative exploration of the model for understanding success in quality (MUSIQ). BMC Health Serv Res. 2018;18:584.CrossRef Reed JE, Kaplan HC, Ismail SA. A new typology for understanding context: qualitative exploration of the model for understanding success in quality (MUSIQ). BMC Health Serv Res. 2018;18:584.CrossRef
32.
go back to reference Artino AR, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational research: AMEE guide no. 87. Med Teach. 2014;36:463–74.CrossRef Artino AR, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational research: AMEE guide no. 87. Med Teach. 2014;36:463–74.CrossRef
33.
go back to reference Hsieh H‑F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.CrossRef Hsieh H‑F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.CrossRef
34.
go back to reference Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. Washington: National Academy Press; 2001. Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. Washington: National Academy Press; 2001.
36.
go back to reference Badiru AB, Thomas MU. Quantification of the PICK Chart for process improvement decisions. J Enterp Transform. 2013;31:1–15.CrossRef Badiru AB, Thomas MU. Quantification of the PICK Chart for process improvement decisions. J Enterp Transform. 2013;31:1–15.CrossRef
37.
go back to reference Bourne E, Short K, McAllister L, Nagarajan S. The quantitative impact of placements on allied health time use and productivity in healthcare facilities: a systematic review with meta-analysis. Health Prof Educ. 2019;20:8–40. Bourne E, Short K, McAllister L, Nagarajan S. The quantitative impact of placements on allied health time use and productivity in healthcare facilities: a systematic review with meta-analysis. Health Prof Educ. 2019;20:8–40.
38.
go back to reference Waters L, Lo K, Maloney S. What impact do students have on clinical educators and the way they practise? Adv Health Sci Educ. 2018;23:611–31.CrossRef Waters L, Lo K, Maloney S. What impact do students have on clinical educators and the way they practise? Adv Health Sci Educ. 2018;23:611–31.CrossRef
39.
go back to reference Kemp C, van Herwerden L, Molloy E, et al. How do students offer value to organisations through work integrated learning? A qualitative study using social exchange theory. Adv Health Sci Educ. 2021;26:1075–93.CrossRef Kemp C, van Herwerden L, Molloy E, et al. How do students offer value to organisations through work integrated learning? A qualitative study using social exchange theory. Adv Health Sci Educ. 2021;26:1075–93.CrossRef
Metagegevens
Titel
Bridging medical education goals and health system outcomes: An instrumental case study of pre-clerkship students’ improvement projects
Auteurs
Bridget C. O’Brien
Josué Zapata
Anna Chang
Edgar Pierluissi
Publicatiedatum
08-04-2022
Uitgeverij
Bohn Stafleu van Loghum
Gepubliceerd in
Perspectives on Medical Education / Uitgave 4/2022
Print ISSN: 2212-2761
Elektronisch ISSN: 2212-277X
DOI
https://doi.org/10.1007/s40037-022-00711-1

Andere artikelen Uitgave 4/2022

Perspectives on Medical Education 4/2022 Naar de uitgave