Skip to main content
Top
Gepubliceerd in: Perspectives on Medical Education 3/2015

Open Access 01-06-2015 | Short Communication

Feedback on role model behaviour: effective for clinical trainers?

Auteurs: H.G.A. Ria Jochemsen-van der Leeuw, Margreet Wieringa-de Waard, Nynke van Dijk

Gepubliceerd in: Perspectives on Medical Education | Uitgave 3/2015

share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail
insite
ZOEKEN

Abstract

Aim

The aim of this study was to assess changes in role model behaviour of clinical trainers after giving personal feedback.

Methods

First-year general practitioner (GP) trainees at two institutes for GP speciality training in the Netherlands were asked to complete an assessment of their clinical trainers: the Role Model Apperception Tool (RoMAT). The RoMAT consists of attributes of positive role modelling divided into two components (Caring Attitude and Effectiveness) and was scored on a 5-point Likert scale twice. After the first assessment moment, the trainers received their personal scores combined with the mean score of their peers. The trainers were divided into three performance groups: below average, average and above average.

Results

Only the group with the lowest scores showed an improvement on the Effectiveness component of the RoMAT from 3.89 to 4.08 (p = 0.04) with an effect size of.52, showing a large effect. This pattern is confirmed by the number of trainers shifting from the below average performance group to the average (7) and above average (5) performance groups.

Conclusion

Giving feedback to clinical trainers resulted in better scores on the Effectiveness characteristics. This indicates that role model behaviour of clinical trainers can be improved.

Background

In clinical practice, trainees develop into competent physicians by working alongside clinical trainers. Besides being teachers, mentors or coaches, clinical trainers are also observed and imitated as role models when, for example, they are providing patient care or working together with other healthcare workers. To use clinical workplace learning as an educational tool to foster the development of trainees into competent professionals, it is therefore important to enhance the role model behaviour of clinical trainers [1].
To assist in this effort, we have developed and validated a tool to assess role model behaviour. This Role Model Apperception Tool (RoMAT) [2] consists of attributes of positive role modelling drawn from a systematic review of the literature [3]. Trainees can use the RoMAT to distinguish, through apperception, between positive and negative role modelling, and to assess clinical trainers’ role model performance. The RoMAT (See Appendix) consists of 17 items scored on a 5-point Likert scale and divided over two components: ‘Caring Attitude’ and ‘Effectiveness’. ‘Caring Attitude’ clusters items that reflect characteristics of the relationship between trainers and their patients, trainees and others. ‘Effectiveness’ represents items relating to the ability of trainers to provide their patients and trainees with what they need. Both components include an equal number of items addressing personal, teaching and clinical qualities, with high reliabilities [2].
Maker and colleagues [4] showed a positive effect of personal feedback on role model behaviour of faculty members of a surgical department. Van Es and colleagues [5] described recommendations for improving acceptance of feedback, for instance by providing trainers with the score ranges of their colleagues, especially if combined with narrative comments.
The aim of this study was to replicate the results of Maker and colleagues [6] while taking into account the recommendations of Van Es et al., by assessing changes in role model behaviour of GP trainers, using the RoMAT to give personal feedback on the role model function.

Methods

In the Netherlands, trainees spend 3–4 days a week during their first and third years of GP speciality training working at GP practices under the supervision of their GP trainers. Throughout their training, trainees spend one day a week at one of the eight institutes for GP speciality training, where they are instructed by teachers. Each year, GP trainers attend eight training days at these institutes.
We invited first-year GP trainees, starting between September 2012 and March 2013 at two institutes—the Academic Medical Center of the University of Amsterdam (AMC-UvA) (n = 91) and the VU University Medical Center Amsterdam (VUMC) (n = 79)—to participate in our study. We asked the respondents to complete the RoMAT for their trainers at 6 months (T1) and 12 months (T2) after the start of their first training year as part of the bi-annual evaluation of the trainers. Each trainee rated one trainer. In case of training practice with several trainers, trainees were asked to rate their main trainer. All respondents were informed that this newly introduced tool would be part of a study, that participation in the study was voluntary and that questionnaires would be coded in order to prevent responses being traceable to individual respondents. Ethical approval was sought and granted by the Dutch Association for Medical Education (NVMO file number 226).
After the first assessment, we calculated means and SD scores for the overall scales and of each item separately for all trainers at the same institute. As written feedback, the trainers received their personal scores combined with the mean score of their peers. Also, trainees were encouraged to discuss the scores they noted with their trainers.
We divided the trainers into three performance groups for each of the components (i.e. Caring Attitude and Effectiveness) based on their T1 scores: below average performance (≥ 1 SD below the mean), average performance (− 1 SD <mean <+ 1 SD) and above average performance (≥ 1 SD above the mean). For each group we compared the scores of both components separately at T1 and T2 with a paired t-test and calculated the effect sizes (ES) by dividing the mean difference by the common SD. A p-value < 0.05 was considered a statistically significant change; an ES of < 0.3 was regarded as small, between 0.3 and 0.5 as moderate, and > 0.5 as large. We also calculated the number of trainers in each group (below, mean and above average) before and after feedback.

Results

A total of 76 trainees responded at both T1 and T2, namely 68 (67 %) trainees at the AMC–UvA and 8 (11 %) at the VUMC (Table 1), resulting in 76 available scores of trainers. The average score of all participants on the Caring Attitude component at T1 was 4.63 (SD 0.36) and at T2 4.62 (SD 0.37). On the Effectiveness component the score at T1 was 4.40 (SD 0.38) and at T2 4.41 (SD 0.42). No significant change in scores on the Caring Attitude component was established for the three groups. The scores on the Effectiveness component in the below average performance group increased from 3.89 to 4.08 (p = 0.04) with an ES of.52, showing a large effect. This pattern was confirmed by the number of trainers shifting from the below average performance group to the average (7) and above average (5) performance groups.
Table 1
Results for the three groupsa before (T1) and after (T2) feedback for both components of the Role Model Apperception Tool (RoMAT)
Component of RoMAT
 
Group 1 (≥ 1 SD below the mean)
Group 2 (− 1 SD <mean < 1 SD)
Group 3 ( ≥ 1 SD above the mean)
Caring Attitude
Mean score (SD) T1
4.01 (0.14)
4.71 (0.16)
5.00 (0.00)
Mean score (SD) T2
4.15 (0.32)
4.71 (0.25)
4.82 (0.38)
P
0.134
0.932
0.088
ESb
0.41
0.01
0.47
Number of trainers at T1/T2
15/13
46/51
15/12
Effectiveness
Mean score (SD) T1
3.89 (0.11)
4.46 (0.21)
4.96 (0.07)
Mean score (SD) T2
4.08 (0.37)
4.45 (0.38)
4.75 (0.36)
P
0.04 c
0.92
0.10
ESb
0.52
0.02
0.55
Number of trainers at T1/T2
18/11
47/49
11/16
aThe classification of the trainers in three performance groups for both components (Caring Attitude and Effectiveness): (1) Below average performance (≥ 1 SD below the mean), (2) Average performance (− 1 SD < mean <+ 1 SD) and (3) Above average performance (≥ 1 SD above the mean)
bEffect Size (ES) = Meandifference/SDdifference [8]
cSignificant difference at p < 0.05

Discussion

After personal feedback, only the group of GP trainers with the lowest scores showed an improvement on the Effectiveness component of the RoMAT. This pattern is confirmed by the number of trainers shifting from the group with below average performance to the average and above average performance groups. The improvement on the Caring Attitude component was not statistically significant, maybe due to the occurrence of a ceiling effect. Trainers received high scores on the Caring Attitude at the start of a traineeship, probably because GP trainers, in contrast to many clinical specialists, voluntarily choose to become a trainer and are thus very motivated to welcome trainees. Attributes of the Caring Attitude component represent this behaviour. In contrast, the Effectiveness scores in comparison to the Caring Attitude scores at T1 start low and increase; in the below average performance group, this increase is significant over the second 6-month period, showing a large effect at T2. In a previous study, trainees just starting their traineeships also rated their clinical trainers with higher scores on the Caring Attitude component than on the Effectiveness component, suggesting that, when starting their traineeship, they are most sensitive to the aspects of the Caring Attitude component and have not yet discovered possible negative characteristics of their trainers. Trainees seem to focus more on attributes of the Effectiveness component as they progress through their traineeship, indicating that trainers with higher scores on the Effectiveness component are better role models for preparing trainees to become independent GPs [6]. In other words, the aspects valued by trainees seem to shift over time, with aspects related to a safe learning environment being more important in the beginning and aspects related to becoming an independent GP being more important in a later phase. Nevertheless, in the groups with above average performance, the scores were already high on both components at the start, resulting in regression to the mean at T2. Although all scores show the same pattern and regression to the mean is to be expected to influence all results, only the improvement of the Effectiveness score in group 1, the low performance group, showed a significant rise with a large effect, confirmed by the largest shift of trainers from one group to the other.
The results of our study are consistent with those of Maker and colleagues with regard to the improvement in trainers with the lowest scores and in the number of trainers shifting from the lowest to the highest scores [4]. However, there are two important differences. Firstly, the surgical trainers also improved their overall score. Secondly, the surgical trainers showed improvement on four attributes that are similar to attributes of both the Caring Attitude and the Effectiveness component, while GP trainers only improved on the characteristics of the Effectiveness component. These differences might originate from contrasting conditions for GP and surgical trainers. GP trainers are volunteers and have to attend eight training days each year, so they are highly motivated and already focussed on being trainers. Furthermore, GP trainees have only one or two GP trainers as their role models at any one time, making it difficult to compare and to distinguish between positive and negative role modelling, while they also have to discuss their feedback with their trainers. This possibly results in high scores at the start of their training. Surgical trainees, working alongside surgeons who at the same time function as their trainers, have more opportunities to make comparisons and their assessments are anonymous. They therefore tend to give their trainers lower scores, resulting in more opportunities for an increase in scores.
A more recent study by Maker and colleagues showed that trainers with the lowest evaluations even further improve towards being a role model by way of a detailed feedback letter summarizing trainer performance, compared with role models (trainers with the highest performance scores) [7].

Limitations

There was a low response in completing the RoMAT for the second time at one of the institutes, probably because the form was not sent off together with the standard evaluation, but in a separate email with only one reminder and because participation was voluntary. Bias can be expected as a result of the low response; trainees may not have responded in fear of the consequences for their evaluation by the trainer at the end of their training year. It may be that trainees who did respond were more positive about their trainers, which may have restricted the effect.
Further research with higher response rates is needed. To attain higher response rates, it might be helpful to implement the RoMAT, also in clinical settings, and to monitor both trainees’ and trainers’ results.

Implications for future education

This study indicates that the RoMAT can be used to evaluate the effectiveness of educational interventions that are designed to improve role model behaviour of trainers.

Conclusion

Giving feedback to trainers resulted in better scores on the Effectiveness component of the RoMAT for trainers from the below average performance group. This outcome seems to indicate that trainees are able to use the RoMAT to distinguish between positive and negative role modelling, and that the role model behaviour of GP trainers can be improved.
Funding/Support
This study was funded by CASH (Committee for Activities to promote the Education of General Practitioners).

Acknowledgements

The authors are sincerely grateful to H. Breuker (VUMC), the staff of the Secretariat of the institute for GP specialty training (AMC–UVA) and N.J.R. Luijten for their assistance in collecting and processing the data for this study.

Other disclosures

None.

Ethical approval

This study (number 235) was approved by the Ethical Review Board of the NVMO (Dutch Association for Medical Education).
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail
Bijlagen

Appendix. The Role Model Apperception Tool (= RoMAT) [3]

 
CA/EFa
My Clinical Trainer…
Strongly disagree
Disagree
Neutral
Agree
Strongly agree
1
EF
Has excellent clinical reasoning skills
1
2
3
4
5
2
CA
Conveys empathy for patients
1
2
3
4
5
3
CA
Communicates well with patients and relatives
1
2
3
4
5
4
EF
Understands learners’ needs and is committed to the growth of learners
1
2
3
4
5
5
CA
Establishes rapport with learners
1
2
3
4
5
6
CA
Has a positive attitude towards learners
1
2
3
4
5
7
CA
Demonstrates enthusiasm for one’s work
1
2
3
4
5
8
CA
Is patient
1
2
3
4
5
9
CA
Has a positive interaction with other healthcare workers
1
2
3
4
5
10
EF
Makes learning exciting and stimulating
1
2
3
4
5
11
EF
Has self-confidence
1
2
3
4
5
12
CA
Is available to learners
1
2
3
4
5
13
CA
Is honest and has integrity
1
2
3
4
5
14
EF
Has leadership qualities
1
2
3
4
5
15
EF
Is aware of his/her role model status
1
2
3
4
5
16
CA
Is nice and easy to work with
1
2
3
4
5
17
EF
Is professionally competent in difficult clinical situations and able to cope with adversity
1
2
3
4
5
aComponents of the RoMAT: Caring Attitude (CA) and Effectiveness (EF)
Literatuur
1.
go back to reference Benbassat J. Role modelling in medical education: the importance of a reflective imitation. Acad Med. 2014;89(4):1–5.CrossRef Benbassat J. Role modelling in medical education: the importance of a reflective imitation. Acad Med. 2014;89(4):1–5.CrossRef
2.
go back to reference Jochemsen-van der Leeuw HGA, Dijk N van, Wieringa-de Waard M. Assessment of the clinical trainer as a role model; a role model apperception tool (RoMAT). Acad Med. 2014;89(4):671–7.PubMedCrossRef Jochemsen-van der Leeuw HGA, Dijk N van, Wieringa-de Waard M. Assessment of the clinical trainer as a role model; a role model apperception tool (RoMAT). Acad Med. 2014;89(4):671–7.PubMedCrossRef
3.
go back to reference Jochemsen-van der Leeuw HGA, Dijk N van, Etten-Jamaludin FS van, Wieringa-de Waard M. The attributes of the clinical trainer as a role model: a systematic review. Acad Med. 2013;88:26–34.PubMedCrossRef Jochemsen-van der Leeuw HGA, Dijk N van, Etten-Jamaludin FS van, Wieringa-de Waard M. The attributes of the clinical trainer as a role model: a systematic review. Acad Med. 2013;88:26–34.PubMedCrossRef
4.
go back to reference Maker VK, Curtis KD, Donnelly MB. Faculty evaluations: diagnostic and therapeutic. Curr Surg. 2004;61(6):597–601.PubMedCrossRef Maker VK, Curtis KD, Donnelly MB. Faculty evaluations: diagnostic and therapeutic. Curr Surg. 2004;61(6):597–601.PubMedCrossRef
5.
go back to reference Es J Van, Visser MRM, Wieringa-de Waard M. Do GP trainers use feedback in drawing up their Personal Development Plans (PDPs)? Med Teach. 2012;43:e718–24. Es J Van, Visser MRM, Wieringa-de Waard M. Do GP trainers use feedback in drawing up their Personal Development Plans (PDPs)? Med Teach. 2012;43:e718–24.
6.
go back to reference Jochemsen-van der Leeuw HGA, Dijk N van, Jong W de, Wieringa-de Waard M. Educating the clinical trainer: professional gain for the trainee? A controlled intervention study in General Practice. Perspect Med Educ 2014;3:455–73. doi:10.1007/s40037-014-0142-zPubMedCentralPubMedCrossRef Jochemsen-van der Leeuw HGA, Dijk N van, Jong W de, Wieringa-de Waard M. Educating the clinical trainer: professional gain for the trainee? A controlled intervention study in General Practice. Perspect Med Educ 2014;3:455–73. doi:10.1007/s40037-014-0142-zPubMedCentralPubMedCrossRef
7.
go back to reference Maker VK, Lewis JL, Donnely MB. Ongoing faculty evaluations: developmental gain or just more pain? Curr Surg. 2006;63:80–4.PubMedCrossRef Maker VK, Lewis JL, Donnely MB. Ongoing faculty evaluations: developmental gain or just more pain? Curr Surg. 2006;63:80–4.PubMedCrossRef
8.
go back to reference Hojat M, Xu G. A visitor’s guide to effect sizes. Adv Health Sci Educ. 2004;9:241–9.CrossRef Hojat M, Xu G. A visitor’s guide to effect sizes. Adv Health Sci Educ. 2004;9:241–9.CrossRef
Metagegevens
Titel
Feedback on role model behaviour: effective for clinical trainers?
Auteurs
H.G.A. Ria Jochemsen-van der Leeuw
Margreet Wieringa-de Waard
Nynke van Dijk
Publicatiedatum
01-06-2015
Uitgeverij
Bohn Stafleu van Loghum
Gepubliceerd in
Perspectives on Medical Education / Uitgave 3/2015
Print ISSN: 2212-2761
Elektronisch ISSN: 2212-277X
DOI
https://doi.org/10.1007/s40037-015-0184-x

Andere artikelen Uitgave 3/2015

Perspectives on Medical Education 3/2015 Naar de uitgave

The Writer’s Craft

The art of limitations