Figures
Abstract
Objective
Previous studies identified different typologies of role models (as teacher/supervisor, physician and person) and explored which of faculty's characteristics could distinguish good role models. The aim of this study was to explore how and to which extent clinical faculty's teaching performance influences residents' evaluations of faculty's different role modelling statuses, especially across different specialties.
Methods
In a prospective multicenter multispecialty study of faculty's teaching performance, we used web-based questionnaires to gather empirical data from residents. The main outcome measures were the different typologies of role modelling. The predictors were faculty's overall teaching performance and faculty's teaching performance on specific domains of teaching. The data were analyzed using multilevel regression equations.
Results
In total 219 (69% response rate) residents filled out 2111 questionnaires about 423 (96% response rate) faculty. Faculty's overall teaching performance influenced all role model typologies (OR: from 8.0 to 166.2). For the specific domains of teaching, overall, all three role model typologies were strongly associated with “professional attitude towards residents” (OR: 3.28 for teacher/supervisor, 2.72 for physician and 7.20 for the person role). Further, the teacher/supervisor role was strongly associated with “feedback” and “learning climate” (OR: 3.23 and 2.70). However, the associations of the specific domains of teaching with faculty's role modelling varied widely across specialties.
Citation: Boerebach BCM, Lombarts KMJMH, Keijzer C, Heineman MJ, Arah OA (2012) The Teacher, the Physician and the Person: How Faculty's Teaching Performance Influences Their Role Modelling. PLoS ONE 7(3): e32089. https://doi.org/10.1371/journal.pone.0032089
Editor: Giuseppe Biondi-Zoccai, Sapienza University of Rome, Italy
Received: December 22, 2011; Accepted: January 22, 2012; Published: March 12, 2012
Copyright: © 2012 Boerebach et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study is part of the research project “Quality of clinical teachers and residency training programs” which is co-financed by the Dutch Ministry of Health, the Academic Medical Center, Amsterdam, and the Faculty of Health and Life Sciences of the University of Maastricht. BCMB, KL, CK and MJH are employed by the Academic Medical Centre, Amsterdam. OAA is a recipient of a Veni grant (#916.96.059) from the Netherlands Organization for Scientific Research (NWO). The funders had no role in study design, data collection, data analysis, data interpretation, decision to publish or writing of the report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
An important part of the learning process of residents occurs through observation and imitation of more experienced faculty, who act as role models. The importance of good role modelling in residency training is globally understood and is believed to be an important teaching method in shaping the values, attitudes, behaviour, and ethics of residents [1]–[3]. Role modelling can be seen as an overarching activity that encompasses everything faculty do in their being and acting as professionals [1]. Insight in methods to improve good role modelling could be of great interest to clinical faculty.
Previous studies identified different components of role modelling [4] resulting in various typologies of role models as a teacher/supervisor, physician and person (see box S1) [5]. Equally, the specific characteristics of role models can be categorized into three different categories: clinical qualities, teaching qualities and personal qualities [1], [6]. However, the relationship between the distinctive roles and role model's characteristics is largely unknown (figure 1). There seems to be no one-on-one relationship, as some clinical and personal qualities have been shown to influence all distinctive roles simultaneously [7] and teaching qualities have been shown to influence the physician role [8]. These findings raise the question how the distinctive roles are influenced by different role model characteristics.
Although a few empirical studies identified some teaching qualities that could influence role modelling [3], [8], [9], these studies do not distinguish between the various role modeling typologies. Besides they do not study differences across specialties. The aim of this study is to provide clinical faculty with specific insight in how their teaching performance could influence their being seen as different kinds of role models by residents. More specifically, this study wants to explore 1) through which of the distinctive roles teaching performance influence role modelling, 2) if this occurs similarly across specialties. To answer these research questions, we used the Systematic Evaluation of Teaching Qualities (SETQ) system to obtain faculty's teaching performance data [10]–[12].
Materials and Methods
Study Population and Setting
To address the objective of this study, we gathered quantitative empirical data. We collected the data using web-based questionnaires filled out by residents. Data were collected between September and November 2010. In total 317 residents of 17 different residency training programs (in Anesthesiology, Internal medicine, Obstetrics & Gynecology, Pediatrics and Surgery) were invited to fill out the questionnaires. The residency programs were situated in three academic (eight programs) and eight non-academic (nine programs) teaching hospitals.
Residency training in The Netherlands is a joint responsibility of several teaching faculty who form an educational team. The educational teams of the residency training programs included in this study ranged from 6 to 87 faculty per team. In some larger educational teams, smaller sub-teams are formed to guarantee the personal interaction between faculty and residents. At the sub-teams of pediatrics, surgery and internal medicine, these sub-teams may also focus on different subspecialties. Because of this setting, residents could choose which and how many faculty to evaluate, based on whose teaching performance the resident felt he or she was able to evaluate accurately. In total, 441 faculty could be evaluated. Participants were invited to participate via email. The invitation email mentioned the formative purpose and use of the evaluations and stressed the confidential and voluntary character of participation.
Study design and Questionnaires
For measurement of faculty's teaching performance, the System for Evaluation of Teaching Qualities (SETQ) questionnaires, which are based on the Stanford Faculty Development Program (SFDP-26) questionnaires [13], were used. SETQ is a dynamic system developed for the continuous evaluation and development of faculty involved in teaching residents and is widely used in The Netherlands [11]. The questionnaires are developed and validated for different specialties and evaluate faculty's teaching performance in five domains of teaching: learning climate, professional attitude towards residents, communication of goals, evaluation of residents and feedback [10]–[12]. All questionnaires contain 20 “generic items”, for some specialties the questionnaires contain additional items (see appendix S1). To obtain reliable SETQ evaluation data on each faculty (predicted Cronbach's alpha of all individual domains >0.70), at least six resident evaluations are needed for anesthesiology [11], five for internal medicine and pediatrics [10], four for obstetrics and gynecology [12] and seven for surgery (unpublished study).
To evaluate faculty's role modelling, additional questions were formulated after discussing faculty's role modelling with a group of 15 anesthesiology residents. Based on the literature we initially proposed a four role model typology (physicians, person, teacher and supervisor) [5], [9] and discussed these typologies as described in the literature. Because in residents' perception the roles teacher and supervisor could not be adequately distinguished in their daily residency training, the initial four role model typology was than reduced to contain only three role models. A similar classification has been used in previous studies [4], [7], [14].
Outcome variables
The outcome variables were residents' perception of faculty's role modelling on the three different role models typologies. At the end of the SETQ questionnaire each respondent was asked to answer three questions about faculty's role modelling. These items were: During my residency, this faculty is a role model to me in his/her role as… (Q1: teacher/supervisor, Q2: physician, Q3: a person). The items were scored on 5-point Likert scale: 1 = “strongly disagree”, 2 = “disagree”, 3 = “neutral”, 4 = “agree”, 5 = “strongly agree”, and there was an additional option “I can not judge”. Each item was preceded by some examples of typical skills and characteristics for this role model typology (see box S2).
Main predictors
The main predictor was faculty's teaching performance, as evaluated by residents via the SETQ questionnaires. In the analyses, we included faculty's overall teaching performance, which is defined as the mean score of all the SETQ items (appendix S1), as a predictor. We also included faculty's teaching performance on the five previously defined SETQ domains as predictors (see figure 2). The items of the SETQ questionnaires were scored on the same 5-point Likert scale as the role model items.
Covariates
The analytical models used to analyze the associations between the predictors and the outcome variables, were adjusted for several covariates: faculty's sex and experience, resident's sex and residency year (all models) and hospital and specialty (only in the models where more than one department and/or specialty was included) [15]. We performed the analyses both adjusted and unadjusted for these covariates to explore if they confounded the associations between teaching performance and the role model typologies.
Analytical Strategies
We were aware that cross-clustering could affect the associations between teaching performance and the role modelling typologies, because in both the evaluating (residents) and evaluated (faculty) participants, there was potential clustering. More specifically, if faculty were evaluated by more then one resident in their residency program, the associations could be stronger within that individual faculty than across faculty members. Additionally, associations of scores given by the same resident, could be stronger compared to associations of scores given across residents. Because the data contained both nominal and continuous variables, we used ordinal logistic generalized estimating equation (GEE) models to study associations between faculty's teaching performance and the role model typologies. These regression equations can analyze both nominal and continuous variables simultaneously and allow for appropriate adjustment of cross-clustering.
However, before tackling our main objectives, we used descriptive statistics to describe the participants' characteristics. Besides, we described median and mean scores of residents' ratings of faculty on the three role model items.
To explore the association between faculty's overall teaching performance and the distinctive type of role models we used GEE models with faculty's overall teaching performance as predictor and the role modelling items as outcome variables. Similarly, we used GEE models with faculty's performance on specific domains of teaching as predictors, to explore if these domains influenced the role model typologies. Estimated odd ratios and their 95% confidence interval were reported. Data were analyzed using statistical package PASW Statistics 18.0.2 for windows (SPSS Inc., 2009).
Results
In total, 219 (69%) residents filled out 2111 questionnaires about 423 (96%) faculty (see table 1 for participants' characteristics). The residents who participated in this study were equally divided over different phases of their residency training (1st year: 19%, 2nd: 14%, 3rd: 16%, 4th: 24%, 5th: 17%, 6th: 9%).
Further, the median and mean scores given by the residents on the different role model typologies indicate that residents rate their teaching faculty higher on the physician role, compared to the teacher/supervisor and person role (table 2).
The odd ratios for the adjusted associations between overall teaching performance and the different type of role models were consistently highest for the role of teacher/supervisor over all specialties (OR: from 47.3 to 166.2), see table 3. The odd ratios for the adjusted associations between the specific domains of teaching and the role modelling items differed per domain, type of role model and specialty (see table 4). Overall, professional attitude towards residents and feedback had the highest odd ratios for the teacher/supervisor the person role, while professional attitude towards residents and evaluation of residents had highest odd ratios for the physician role. However, there were considerable differences across specialties. For all analyses, the unadjusted models differed only marginally from the adjusted models, so only the results of the adjusted models are shown (table 3 and 4).
In general the odd ratio represents the chance that a faculty improves by one point on the outcome variable (being seen as a role model), if he/she improves by one point on the predictor variable (teaching performance). For example, the odd ratio for the adjusted association between overall teaching performance and the role of physician for anesthesiology faculty is 9.7. This signifies that the chance is 9.7∶1 that this faculty is being regarded as a better physician role model by the residents, if this faculty improves his overall teaching performance by one point. Further we point out that these odd ratios, like odd ratios in general, have a logistic scale. Consequently, the ostensible wide variation in odd ratios in table 3, represent a considerably lower amount of variation when one should recalculate them into chance percentages.
Discussion
Main Findings
In the continuous search for better understanding and improving role modelling as a teaching strategy in residency training, a clear perspective is needed on the determinants of clinical faculty being perceived by residents as a role models. This study was set out to explore if faculty's role modelling is influenced by their teaching performance and if this occurs similarly across specialties. The results of this study present empirical evidence of the great influence of faculty's overall teaching performance on being seen as a role model teacher/supervisor, physician and person. Further, we found that the influence of the specific domains of teaching on the role modelling typologies varied widely across specialties. Overall, professional attitude towards residents and feedback were the strongest predictors of the teacher/supervisor and the person role, while professional attitude towards residents and evaluation of residents were the strongest predictors of the physician role.
Strengths and Limitations of this Study
The multicenter approach that included both academic and non-academic teaching hospitals and the fact that close to 100% of the faculty of the educational teams included in this study were evaluated, imply that the study population represents a valid sample of Dutch clinical faculty. Further, the observed numbers of residents' evaluations completed per faculty were adequate for sufficient reliability of the SETQ evaluation data for anesthesiology, obstetrics and gynecology and surgery [11], [12]. For internal medicine and pediatrics, the numbers of evaluations completed per faculty, although, on average, lower than five, were close to the recently reported analysis that showed that 2 to 4 evaluations per faculty could still yield high reliability data (Cronbach's alpha >0.88) [10], [12]. We must also be cautious in transferring the results to other specialties or health care systems [16]. Note that for anesthesiology and internal medicine only faculty of one academic hospital were included in the study.
Further, the presumed causal relationship between teaching performance and role modelling was based on theory [1]. However, we cannot exclude the possibility that the associations found in this study are caused by a reverse relationship between teaching performance and role modelling or by a triangular relationship with a variable we did not measure in this study.
Explanation of the Results
The results regarding faculty's overall teaching performance are in line with previous studies that suggest improved role modelling when faculty's teaching performance was enhanced [1], [3], [17].
Some domains of teaching performance simultaneously influenced residents' perception of all faculty's role model types, while others had a positive influence on just one specific role model type, while having hardly no influence on another role model type. An explanation may be the definitions of the domains of teaching performance: For example, the domain professional attitude towards residents includes approachability, listening attentively and being respectful toward residents, a broad definition [11]. Hence, it is perhaps not surprising that it is generally the most influential predictor across all three role modelling types. The description of evaluation of residents by contrast [11], includes performances that are clearly more prone to influence the role of teacher/supervisor or physician compared to the person role [5], [9]. The negative relationships between some domains of teaching and the role modelling typologies of physician and person should be interpreted with some caution, given the large confidence intervals that include both positive and negative relationships. If there would be a negative relationship however, it is possible that residents regard faculty who regularly evaluate residents or who focus on the learning aspects of the residency training as more demanding faculty. This may explain why residents at some specialties report a negative relationship between the domain evaluation of residents and the role model type of person.
The differences across specialties as found in this study may result from the fact that the core competences that have to be learned and the way training is organized differs. In surgical residency training for example, residents spend a considerable amount of time in individual training in the operation theater to acquire appropriate technical skills. Direct performance feedback to residents will play a prominent role in this setting. Feedback provided during an operation in the operation theatre may have more impact on residents, compared to feedback given in more quiet settings. This hypothesis supports the finding that surgical residents regard feedback as a stronger influencer of the teacher/supervisor and the person role compared to other specialties.
Beside the differences in residency training programs, previous studies suggest that residents across specialties cannot be considered a homogeneous group [18]–[20]. The kind of role models residents are looking for could vary as a result of differences in residency training programs or residents' personality, interests or career motivation.
Implications for Clinical Education, Research and Policy
In general the findings of this study underline the importance of the specialty specific context and show that role modelling cannot be regarded as a universal process across specialties. To improve good role modelling, teaching faculty can modify their teaching performance towards the specific goals they have set for themselves and the specific context they teach in. The good news is that many domains of teaching performance evaluated in this study are cognitive in nature, so they can be learned or improved by faculty who want to improve their role modelling as a teaching strategy. A logical first step for faculty who want to improve their teaching performance is to get insight into their current teaching performance. Valid and useful systems that could inform self-assessment could be of great interest to those faculty [21]. Future research could explore if teaching faculty who want to improve their role modelling as a teaching strategy, can succeed by improving their teaching performance.
Conclusions
This study might help teaching faculty in understanding their role modelling better. Based on the reported findings a noteworthy recommendation could be that faculty should consider investing in enhancing specific domains of teaching performance as these domains are proven to be most influential in being seen as a specific type of role model by residents. The reported cross-specialty variations stress the complex processes of role modelling and highlight the importance of the specialty-specific context.
Supporting Information
Appendix S1.
Overview of the items and scales of the SETQ questionnaires. The items shared the same subject “During my residency in [specialty], my attending faculty generally…”. # = this item was in the SETQ questionnaires for Internal Medicine and Gynecology & Obstetrics. ## = this item was in the SETQ questionnaires for Anesthesiology, Pediatrics and Surgery.
https://doi.org/10.1371/journal.pone.0032089.s001
(DOC)
Box S1.
Some characteristics of the role model typologies [5], [9].
https://doi.org/10.1371/journal.pone.0032089.s002
(JPG)
Box S2.
Preceding text of typical role model skills in the role model items of the questionnaires.
https://doi.org/10.1371/journal.pone.0032089.s003
(JPG)
Author Contributions
Conceived and designed the experiments: OAA KL MJH CK. Performed the experiments: MJH CK KL BCMB. Analyzed the data: BCMB OAA. Contributed reagents/materials/analysis tools: KL. Wrote the paper: BCMB KL OAA CK MJH.
References
- 1. Cruess SR, Cruess RL, Steinert Y (2008) Role modelling–making the most of a powerful teaching strategy. BMJ 336: 718–721. 336/7646/718 [pii];10.1136/bmj.39503.757847.BE [doi].
- 2. Paice E, Heard S, Moss F (2002) How important are role models in making good doctors? BMJ 325: 707–710.
- 3. Wright SM, Kern DE, Kolodner K, Howard DM, Brancati FL (1998) Attributes of excellent attending-physician role models. N Engl J Med 339: 1986–1993. 10.1056/NEJM199812313392706 [doi].
- 4. Rauen KC (1974) The clinical instructor as role model. J Nurs Educ 13: 33–40.
- 5. Ullian JA, Bland CJ, Simpson DE (1994) An alternative approach to defining the role of the clinical teacher. Acad Med 69: 832–838.
- 6. Wright SM, Carrese JA (2002) Excellence in role modelling: insight and perspectives from the pros. CMAJ 167: 638–643.
- 7. Ambrozy DM, Irby DM, Bowen JL, Burack JH, Carline JD, et al. (1997) Role models' perceptions of themselves and their influence on students' specialty choices. Acad Med 72: 1119–1121.
- 8. Lombarts KM, Heineman MJ, Arah OA (2010) Good clinical teachers likely to be specialist role models: results from a multicenter cross-sectional survey. PLoS One 5: e15202. 10.1371/journal.pone.0015202 [doi].
- 9. Boor K, Teunissen PW, Scherpbier AJ, van der Vleuten CP, van de Lande J, et al. (2008) Residents' perceptions of the ideal clinical teacher–a qualitative study. Eur J Obstet Gynecol Reprod Biol 140: 152–157. S0301-2115(08)00139-5 [pii];10.1016/j.ejogrb.2008.03.010 [doi].
- 10. Arah OA, Hoekstra JBL, Bos AP, Lombarts K (2011) New Tools for Systematic Evaluation of Teaching Qualities of Medical Faculty: Results of an Ongoing Multi-Center Survey. PLoS One 6: e25983. 10.1371/journal.pone.0025983.
- 11. Lombarts KM, Bucx MJ, Arah OA (2009) Development of a system for the evaluation of the teaching qualities of anesthesiology faculty. Anesthesiology 111: 709–716. 10.1097/ALN.0b013e3181b76516 [doi].
- 12. van der Leeuw R, Lombarts K, Heineman MJ, Arah O (2011) Systematic Evaluation of the Teaching Qualities of Obstetrics and Gynecology Faculty: Reliability and Validity of the SETQ Tools. PLoS One 6: e19142. 10.1371/journal.pone.0019142 [doi];PONE-D-10-05927 [pii].
- 13. Litzelman DK, Westmoreland GR, Skeff KM, Stratos GA (1999) Factorial validation of an educational framework using residents' evaluations of clinician-educators. Acad Med 74: S25–S27.
- 14. Brasher AE, Chowdhry S, Hauge LS, Prinz RA (2005) Medical students' perceptions of resident teaching: have duty hours regulations had an impact? Ann Surg 242: 548–553. 00000658-200510000-00010 [pii].
- 15. Arah OA, Heineman MJ, Lombarts K (2012) Factors Influencing Residents' Evaluations of Faculty Teaching Qualities and Role Model Status. Med Educ. In Press.
- 16. Zibrowski EM, Myers K, Norman G, Goldszmidt MA (2011) Relying on others' reliability: challenges in clinical teaching assessment. Teach Learn Med 23: 21–27. 932282717 [pii];10.1080/10401334.2011.536754 [doi].
- 17. Maker VK, Curtis KD, Donnelly MB (2004) Are you a surgical role model? Curr Surg 61: 111–115. 10.1016/j.cursur.2003.09.003 [doi];S0149794403002186 [pii].
- 18. Buddeberg-Fischer B, Klaghofer R, Abel T, Buddeberg C (2006) Swiss residents' speciality choices–impact of gender, personality traits, career motivation and life goals. BMC Health Serv Res 6: 137. 1472-6963-6-137 [pii];10.1186/1472-6963-6-137 [doi].
- 19. Hoffman BM, Coons MJ, Kuo PC (2010) Personality differences between surgery residents, nonsurgery residents, and medical students. Surgery 148: 187–193. S0039-6060(10)00200-X [pii];10.1016/j.surg.2010.04.005 [doi].
- 20. Hojat M, Zuckerman M (2008) Personality and specialty interest in medical students. Med Teach 30: 400–406. 792376529 [pii];10.1080/01421590802043835 [doi].
- 21. Sargeant J, Armson H, Chesluk B, Dornan T, Eva K et al (2010) The processes and dimensions of informed self-assessment: a conceptual model. Acad Med 85: 1212–1220. 10.1097/ACM.0b013e3181d85a4e [doi].