What we learned in the development of a third-year medical student curricular project

To meet the growing needs of the American population, the National Academy of Science includes teaching quality improvement (QI) and population health as core competencies in medical school training [1]. This training can help students learn how to address health disparities (HD) and eliminate gaps in care to achieve health equity [2, 3]. Our department at an urban, private medical school received educational support to develop a population health curriculum—to teach medical students how to use QI methods to improve care for vulnerable populations. The three objectives of the novel, multifaceted curricula were for students to learn about: (1) QI foundational principles, (2) HD in certain communities and vulnerable populations, and (3) to apply QI processes to improve the care of a vulnerable population.

Using a previously developed QI curriculum already implemented in the Family Medicine clerkship, we developed an accompanying HD curriculum. In this concurrent QI curriculum, students completed an Institute for Healthcare Improvement foundational course and a Plan Do Study Act video lesson. To inform the development of our HD curriculum, we assessed students’ knowledge, attitudes, and perceptions on community-based medicine, patient safety, and care for vulnerable populations to identify curricular and knowledge gaps. The needs assessment survey revealed that most students neither knew how to identify a community or conduct a needs assessment of that community, nor did they feel comfortable addressing HD in various vulnerable populations, specifically among individuals who were incarcerated, homeless, immigrants/refugees, and veterans (Smith RS, Silverio A, Casola AR, et al. Third-year medical students’ self-perceived knowledge about health disparities and community medicine. 2020 [unpublished]). Clerkship directors and medical education team members decided to use the information gathered from the needs assessment survey to create three interactive, online, self-directed HD modules: (1) Introduction to Community Health Needs Assessment, (2) Health and Homelessness, and (3) QI and HD. In addition to completing the modules, students were required to conduct patient chart reviews and design a QI intervention to improve the process of chronic disease management and include an analysis on how this process would impact a vulnerable population of their choice. Students pitched their QI projects to peers and faculty at the end of the clerkship. Throughout the development and implementation of this curriculum, we experienced our share of successes and difficulties and gained critical insight that could be helpful for other medical educators undertaking similar projects.

During this project, we encountered a few surprises.

Learning platforms are critical to successful implementation of educational programs. A key technical challenge was the need to consolidate curricula from two learning platforms into one. The QI curriculum was created on a new platform that the university promoted at the time. We chose a different learning management system for our HD curriculum because it provided a grade center and was the university’s main learning platform. Early in the curriculum development process, it is helpful to be familiar with each of the tools and features of various learning management systems so that you select the one that is most compatible with your educational needs [4].

Curriculum development is not a static process. Bowe and Armstrong proposed a systems-based framework in medical education as a way to use continuous QI to not only monitor individual components and facilitate timely corrections but also to drive innovation and prepare for long-term goals and needs [5]. Based on review of student feedback, we modified lesson content to eliminate redundant information in one lesson (QI and HD). Additionally, we modified the curriculum implementation to accommodate our needs. In the beginning, we implemented the curriculum only with students at the main clinical site, to identify issues and challenges before expansion to other sites. We eventually expanded to two additional sites, where students presented their process maps to site directors. Based on the needs of the clerkship schedule, we changed the format to consolidate all the activities into two days—a “QI boot camp.” We combined the curriculum because many of our students work at various affiliate sites. The boot camp model allowed all students to return to the school to have an introductory lecture, have dedicated time to complete online lessons, work on their group chart review and QI projects, and present their work to peers and faculty. Through continuous feedback and evaluation, we are able to revise and enhance the curriculum to meet current and future needs.

The most important lesson we learned is to assemble the right team from the beginning, including an evaluation expert. Program evaluation is a critical step in curriculum development, as this is needed for ongoing improvement, measuring effectiveness, and assessing the continuous learning of the students [6]. A formal evaluation professional could have helped us with the design of our educational project, selection of evaluation methods and instruments, data collection/analysis, and reporting of results [7]. Continuous assessment of learner feedback is essential for systematic program evaluation [5], which would have allowed us to modify and improve educational components in a timely manner.

Once we had an evaluation member as part of our team, we were able to address procedural and technical flaws. This included using consistent study IDs (preventing multiple completions and allowing the matching of pre-and post-data), changing the release timing of the post-assessment to increase the response rate, implementing periodic checks of our online platform, and adding demographic items to the surveys. Finally, our evaluation expert helped us to assess the quality of our data, suggesting a change to a reflection assignment to obtain robust qualitative data to supplement our quantitative responses.

We learned the importance of evaluating our implementation processes with continuous student feedback. Because we did not build in an ongoing evaluation method, we found out later that website links were no longer up-to-date and that there was redundant material in educational content, so we were able to eliminate one lesson on QI and HD. Based on student feedback about the clarity of our instructions, we streamlined both the curricula and the assessments onto a single platform and added an overview of the project to the clerkship orientation. Assessment of periodic quantitative and qualitative learner feedback is essential to revise a newly implemented curriculum, troubleshoot problems, and evolve the curriculum [5].

Medical educators can utilize focus groups as a helpful, informal way to gather open-ended feedback from students [8] as an alternative to traditional written course evaluations. We recommend a mid-course focus group to identify opportunities to enhance the curriculum in time to be impactful. Our students greatly appreciated the opportunity to provide feedback in this format, and we were able to rapidly develop consensus about course improvements while also discussing overall course improvements to meet the needs of future learners. We have summarized our major lessons learned within the context of Kern’s 6‑step framework for curriculum development in a table in the Electronic Supplementary Material.

While there were challenges to our curriculum rollout, we learned a lot of valuable information about design and implementation. We would emphasize the importance of assembling the right team, as many of the struggles we had could have been avoided if we had had an evaluation expert (in our case, a PhD researcher) and someone familiar with educational electronic platforms working with us from the beginning. We are currently working with our evaluation faculty to ensure we are using the correct processes to collect the most valid data, capture the right outcomes, and develop new research questions and next steps. Moving forward, we realize the benefit of applying a continuous QI process to our curriculum development and implementation. This will help us meet our students’ educational needs, evaluate the curriculum’s impact on the clerkship, and transform medical education for the health care needs of the future.