Pre-clerkship clinical skills and clinical reasoning course performance: Explaining the variance in clerkship performance

Although many medical schools in the United States continue to adhere to a predominantly ‘Flexnerian’ model (two years of basic science courses followed by two years of clinical rotations), several medical schools are undergoing significant curriculum reforms aimed at integrating basic sciences with clinical skills teaching while often reducing the pre-clerkship period to facilitate an early transition to the clinical clerkships [1]. Despite the compression of the pre-clerkship training period, students tend to want more advanced clinical skills training in preparation for their clerkship experience, while faculty remain focused on the basic components of communication skills, patient interviewing, physical exam skills, and clinical reasoning as the foundation for being an effective physician [1, 2]. However, there remains limited consensus on how to deliver pre-clerkship education and, furthermore, many clerkship directors continue to feel that students are not adequately prepared for the clerkship [3, 4].

Regardless, pre-clerkship courses in clinical skills and clinical reasoning have been shown to improve student performance during the clerkships in foundational skills such as communication and development of patient relationships, in addition to student motivational aspects such as educational attitudes, initiative, participation, and dependability [5]. Although isolated pre-clerkship outcomes, such as an objective structured clinical examination (OSCE), are associated with failures on a subsequent OSCE, individual pre-clerkship measures often do not adequately assess the overall impact of pre-clerkship teaching on student clinical performance in the clerkship [5‐7]. Some have suggested that a combination of pre-clerkship assessments may better explain the variance in a student’s clinical abilities on the clerkship [3].

Previous studies have consistently demonstrated that performance on standardized exams is correlated to performance on future standardized exams [8]. Further, the implementation of longitudinal progress testing is able to identify learners with persistent knowledge deficits and reduce licensing exam failures [9, 10]. Since the observation and remediation of medical students’ clinical skills on the clerkship is inconsistent, the early identification of students who may struggle in the clerkship period would be of significant benefit [11, 12]. The pre-clerkship period offers the opportunity for the deliberate practice of clinical skills in a controlled environment (e. g. a simulation centre using standardized patients), which has been found to be a better approach for the remediation of clinical skills deficits [13]. However, the ability to identify students with persistent clinical skills or clinical reasoning deficits in the pre-clerkship period that may persist in the clerkship has proven to be challenging [7, 8]. The integration of the pre-clerkship courses, Introduction to Clinical Medicine (ICM) and Introduction to Clinical Reasoning (ICR), at our institution provided a unique opportunity to examine how integrated pre-clerkship assessments may better explain the variance of student performance in the clerkships, and identify students likely to struggle in the clerkship.

Prior to a comprehensive curriculum re-design at our institution, the leadership of the ICR and ICM courses began to work towards integration in both instruction and evaluation of learners. Previously, the ICM course was primarily focused on teaching and evaluating clinical skills (communication, history taking, and physical exam) while the ICR course was primarily concerned with the development of clinical reasoning, and both courses functioned independently using separate cases and evaluations. The first step towards integration was to develop joint evaluations on the end of pre-clerkship OSCE, in addition to including an oral case presentation as a synthetic evaluation of clinical reasoning within each standardized patient encounter for the ICS course. Additionally, we moved towards integrating course content by using the standardized patient encounters in ICS as the cases for clinical reasoning discussion within the ICR course. Consistent with situativity theory, our goal for integration was to bring together the context and environment of a clinical encounter using a standardized patient so that students were active participants in the learning of both clinical skills and clinical reasoning similar to what they would experience on the clerkship [14]. Even in the early stages of integration at the time of this study, our focus was on the application of skills and knowledge to improve clinical reasoning, and given the emphasis on reducing diagnostic reasoning errors through improvements in health professions education, the early identification of struggling learners represents a significant opportunity for educational interventions [15, 16].

Our study was designed to examine the extent that integrated pre-clerkship course evaluations from ICM and ICR are associated with students’ clinical and standardized test performance across all their clerkships. We hypothesized that these integrated pre-clerkship outcomes would not only demonstrate a strong association with clinical performance on the clerkships and potentially identify at risk students, but also explain a significant amount of the variance in their performance independent of baseline academic abilities.

Students identified as at risk in either the ICM or ICR course were found to have significantly lower scores in both average clerkship grades (2.75 ± 0.36 v. 3.22 ± 0.39, p < 0.001) and on average clerkship NBME scores (69.0 ± 4.61 v. 73.0 ± 5.52, p < 0.001). The effect size as measured by a Cohen’s d was large for both the impact on average clerkship grades (d = 1.25) and average clerkship NBME scores (d = 0.79). In addition, students at risk in both the clinical skills and clinical reasoning courses had a relative risk of 5.85 (95 % CI of 1.88–18.2, p = 0.002) and 6.96 (95 % CI of 3.01–16.1, p < 0.001) for less than passing performance on clerkship grades (Table 1) and on the clerkship NBME subject exams (Table 2), respectively. The multiple linear regression model of students’ performance in ICM and ICR, inclusive of the ICD NBME subject exam, explained 22 % of the variance of the average clerkship NBME subject exam score across all clerkships (Table 3). The ICD NBME subject exam score (β = 0.53, p < 0.0005) and ICR small-group points (β = 0.08, p = 0.014) were significant predictors of the dependent variable. As would be expected, the standardized coefficient, which represents the relative strength of the predictors in the model, of 0.53 for the ICD NBME subject exam indicates a fairly large effect size of 1.25 (95 % CI of 1.11–1.39) on the clerkship NBME subject exam score.

In terms of predicting the variance in average final grade across clerkships, this group of explanatory variables from the ICM and ICR courses accounted for 20.2 % of the variance beyond first-year grade point average (Table 4). All of the following explanatory variables were significant predictors, including ICD NBME subject exam score (β = 0.35, p < 0.0005), ICM preceptor evaluation (β = 0.11, p = 0.001), ICM OSCE score (β = 0.16, p < 0.0005), and ICR small-group points (β = 0.15, p < 0.0005). As opposed to exam scores, each of the predictors had relatively robust standardized regression coefficients translating into effects sizes of 0.75 for the ICD NBME exam, 0.22 for the ICM preceptor evaluation, 0.32 for the ICM OSCE score, and 0.30 for the ICR small-group points.

Our study demonstrates several important findings regarding pre-clerkship clinical assessments. Students identified as at risk in both the ICM and ICR courses (defined as the bottom 13^th percentile) were found to be at significant risk for poor performance on the clerkship. Overall, the integration of clinical skills and clinical reasoning assessments from the pre-clerkship period demonstrate the ability to explain a significant amount of the variance in clerkship performance independent of previous academic abilities. As would be expected, the ICD NBME subject exam is a strong predictor for students’ future performance on clerkship NBME subject exams. However, clinical reasoning ability as measured by student small-group participation is also a significant predictor in the variance of future exam performance. Faculty observation of clinical skills and clinical reasoning using integrated assessments in combination with performance on the end of pre-clerkship OSCE are not just important predictors of student clerkship grades independent of previous academic abilities, but also demonstrate the ability to identify students who are likely to struggle on the clerkship. The fact that students who struggle in these pre-clerkship courses can both be readily identified and demonstrate significantly poorer performance during the clerkship has important implications for educators.

The ability to identify struggling students in the pre-clerkship period is an important aspect of developing and implementing appropriate remediation efforts in a timely fashion [6]. Arguably, using a combination of integrated clinical skills and clinical reasoning assessments to better understand and predict the variance in future clerkship performance has significant advantages over isolated pre-clerkship assessments that may not have adequate sensitivity to predict clerkship performance [7]. Not only can the use of these types of assessments predict overall performance, but specific deficits across a variety of domains (communication skills, professionalism, physical exam skills, or clinical reasoning) may be identified to provide remediation efforts tailored to a student’s individual needs in a controlled setting before moving on to the clerkship. Many schools utilize a clinical skills centre or simulation centre to deliver basic clinical skills and clinical reasoning curriculum in a standardized and relatively protected environment, which is a much more ideal setting for deliberate remediation efforts than allowing students to attempt remediation efforts in the clerkship [13]. In fact, based on the results of this study, our institutions simulation centre in coordination with pre-clerkship clinical skills faculty have coordinated efforts to identify and tailor the remediation of clinical skills and clinical reasoning for medical students before entering the clerkship period.

Our findings suggest that further integration of pre-clerkship courses in clinical skills and clinical reasoning could provide strong predictive power of students’ future clerkship performance, and should be confirmed in a prospective study. In fact, others have promoted the further integration of clinical skills with clinical reasoning where the student can learn both content skills related to accomplishing a basic patient history and physical exam simultaneous with clinical reasoning within the context of a clinical setting [2]. Pre-clerkship curriculum should develop integrated clinical experiences to develop habits of inquiry and opportunities for the application of medical knowledge and clinical skills (patient history and exam) to clinical reasoning [18]. Based upon our findings, we have continued to integrate the ICM and ICR courses, and now have five integrated clinical skill sessions in the pre-clerkship period that have small-group clinical reasoning sessions immediately after standardized patient encounters identical to the context of an ambulatory clerkship setting. Preliminary data indicate that students who perform in the bottom 13^th percentile on these five integrated sessions have a relative risk of 2.02 (95 % CI 1.33–3.06; p < 0.001) of being identified as struggling on the internal medicine clerkship. In addition, we have implemented the use of longitudinal faculty during the pre-clerkship training, which may further improve our ability to identify students at risk for struggling in the clerkship, and provide opportunities for early intervention.

Ultimately, the use of clinical skills centres providing an integrated clinical skills and clinical reasoning curriculum under a controlled environment with input from clerkship directors may be the most beneficial for students, and potentially promote the longitudinal development and vertical integration of these foundational abilities [13].

Our study had several limitations. First, this is a single institution study, and although we had a large number of students in the analysis, our findings may not be generalizable to all other medical schools. Second, we used first year medical school grade point average as a control variable for baseline academic ability in our regression models. Although first year grade point average strongly predicts the outcome measures on the clerkship in our study, it is possible that other academic outcomes from the second year of medical school could have been even more predictive, and lowered the overall predictive power of the ICM and ICR assessments. Additionally, the ICD NBME exam was predictive of clerkship outcomes, especially for future NBME subject exams. However, the standardized regression coefficients for the integrated ICM and ICR outcomes indicate strong contributions of these variables to the overall model, and there is significant construct validity to relate performance on clinical skills and clinical reasoning assessments to these same measures on the clerkship. Of note, the three year longitudinal nature of our study is an important strength, as this represents a fairly large group of students moving through a stable, integrated pre-clerkship curriculum for clinical skills and clinical reasoning.

Our study demonstrates the ability of integrated clinical skills and clinical reasoning assessments to identify at risk students, and explain the variance of students’ future performance in the clerkship, emphasizing the importance of an integrated pre-clerkship curriculum to foster opportunities for early identification and individualized remediation of students before reaching the clerkships. There could be a significant advantage to accurately identify a group of learners struggling in basic clinical skills before entering the clerkship environment, allowing for timely remediation of these critical skills under direct observation in a controlled environment.