Elsevier

Current Surgery

Volume 61, Issue 6, November–December 2004, Pages 597-601
Current Surgery

2004 APDS spring meeting: Part 2
Faculty evaluations: Diagnostic and therapeutic

https://doi.org/10.1016/j.cursur.2004.06.021Get rights and content

Purpose

An excellent teaching program is one that creates an atmosphere of mentorship, leadership, and professional development. The key to all of these may lie in the faculty evaluation. Currently, favorable faculty evaluations are often rewarded, whereas the more critical evaluations are downplayed and seldom used for faculty development. The purpose of this study is to investigate the efficacy of the faculty evaluation as a diagnostic tool to objectively identify individual strengths and weaknesses. Can such a tool then be used as a therapeutic modality to improve and enhance the faculty as a whole?

Methods

Fourth- and fifth-year surgical residents from 1 residency program were asked to collaboratively define 9 characteristics that make a surgical role model. They then anonymously evaluated each of the 44 teaching attendings on each of the 9 criteria. The results of the evaluations, which consisted of the overall mean scores and SD1 for each of the 9 criteria, the attending’s score and SDs above or below the mean, as well as whether the residents considered them to be a Role Model, were sent to each faculty member as an intervention. Six months after the intervention, the residents evaluated the attendings using the same 9 criteria. Evaluations were collected and analyzed following the same procedure as 6 months prior. T-tests and p-values were calculated to determine any significant differences between the 2 data sets.

Results

Based on the results of the pre-intervention evaluations, we were able to delineate faculty members into 3 groups based on their Role Model score. Of the 10 faculty members with the lowest scores (defined as >1 SD below average), 7 improved after the intervention, 3 of them were statistically significant (2 with p < 0.05 and 1 with p < 0.10). In the middle group of 26 faculty members, 1 showed statistically significant improvement (p < 0.05). Interestingly, this was the faculty member with the lowest score in this group. Six faculty originally in the middle group improved to a score greater than 1 SD above average, defining them as a Role Model. The remaining 8 faculty members originally designated as Role Models showed no statistically significant changes in their scores after the intervention. After the intervention, the average score improved for all of the 9 criteria that make a faculty Role Model. Statistically significant improvements were seen in the categories of Provides Feedback (p < 0.05) and Didactic Teaching, Attendance at Didactic activities, and Stimulates Critical Thinking with the Use of Literature (all p < 0.10).

Conclusions

(1) Faculty evaluations are a diagnostic tool with the ability to select out Role Models from the lower scoring groups. (2) After the intervention, the overall average score improved for all of the 9 criteria that make a Role Model, showing improvement of the faculty as a whole. (3) The therapeutic benefits of faculty evaluations were best seen in the faculty with the lowest scores.

Introduction

“Crustaceans take on the tint of minerals in the water they inhabit.” This quote by Rita Charon, MD, may well state that with time, residents begin to resemble the attendings that surround them. Therefore, it is the mentoring faculty that molds the future of surgery. In addition to the faculty, an excellent teaching program is one that creates an atmosphere of mentorship, leadership, and professional development. In order to achieve such a program, one must be able to identify those teaching attendings that are Role Models from those that are not. The key to this may well lie in the faculty evaluation.

Recently, the ACGME has put a major emphasis on faculty evaluations and development of individual faculty members.1 Although faculty evaluations are nothing new, residency programs have traditionally been generous, rewarding the best faculty with teaching awards, and all but ignoring those faculty members with poor evaluations.2, 3 It has been thought that by rewarding the best faculty members, others may be inspired to improve, believing this to be an interventional tool for faculty development. It has been our experience, however, that mediocre faculty frequently do not attend these meetings and find a variety of excuses to deny the excellence of Role Model faculty members.

The purpose of this study was to investigate the efficacy of faculty evaluations as a diagnostic tool to objectively identify individual strengths and weaknesses, and then to determine if this tool can be used as a therapeutic modality to improve and enhance the faculty as a whole. For this purpose, an objective list of criteria, designed by the residents, was created as a template for faculty evaluations.4 The results of the evaluations were then used as a customized interventional tool for the entire faculty. The intent was to improve all faculty members, both individually and as a whole, or at least to provide faculty members a mirror by which to reflect on their performance, eventually leading to faculty development and improvement.

Section snippets

Methods

Twelve fourth- and fifth-year surgical residents from 1 residency program were asked to collaboratively define 9 characteristics that make a surgical role model. The 9 criteria as defined by the residents were didactic teaching, teaching rounds, attendance at didactic activities, demonstrates skills and decision making with confidence and virtuosity, allows [resident] to do procedures according to ability, allows autonomy to make independent decisions, provides feedback, stimulates critical

Results

  • 1

    Faculty Evaluations as diagnostic: The results of the faculty evaluations, and more specifically the Role Model rating, classify the faculty into 1 of 3 groups. The average Role Model rating for the faculty as a whole was 2.28 (out of a possible 3.00) with a standard deviation of 0.40. The Role Model group (Group 1), was defined by a Role Model rating greater than 1 standard deviation above the mean (>2.69). The Average group (Group 2) was defined as within 1 standard deviation above or below

Discussion

The residents in our program felt very strongly that an excellent faculty must demonstrate skills and teaching abilities not only in the operating room, but outside as well. In addition, they should provide feedback and scholarship and demonstrate humanistic qualities. The traditional stereotypical image of a loud, demeaning surgeon, even though academically superior, was quickly rejected by the residents. It is interesting to note that all 9 of the criteria independently chosen by the

Conclusions

  • 1

    Faculty evaluations are a diagnostic tool with the ability to sort out role models from the lower scoring groups.

  • 2

    The overall average score improved for all of the 9 criteria, plus the Role Model category, demonstrating an improvement of the faculty as a whole.

  • 3

    The criteria that showed statistically significant improvements were Provides Feedback, Didactic Teaching, Attendance at Didactic Activities, and Stimulates Critical Thinking with the Use of Literature.

  • 4

    Of the 6 attendings who were more

References (6)

There are more references available in the full text version of this article.

Cited by (15)

  • Four-Year Analysis of a Novel Milestone-Based Assessment of Faculty by General Surgical Residents

    2018, Journal of Surgical Education
    Citation Excerpt :

    Historically, evaluations have flowed from faculty to residents and not in the opposite direction. In order for trainees to feel comfortable evaluating their educators, anonymity is essential.7, 8 Trainees must provide honest feedback without the fear of retaliation.

  • What is Known About the Attributes of a Successful Surgical Trainer? A Systematic Review

    2017, Journal of Surgical Education
    Citation Excerpt :

    Studies that examine the skills required to be an expert trainer often rely on the opinion of the trainer themselves14 rather than the recipients of the training,15 are limited by small numbers,11,16,17 focus on the addition of technology to the surgical environment to improve training,18 and use trainees from single specialties.19 In addition, trainee evaluations of trainers that are critical tend to be downplayed by training program boards20 meaning that poorly performing trainers are less likely to be identified or get the opportunity to review and change their practice. In response to this concern, The Academy of Medical Educators,21 The General Medical Council, and The Royal College of Surgeons of Edinburgh22 have published detailed documents on the expectations they have for surgeons who also act as trainers.

  • Importance of a Training the Trainers (TTT) Course

    2022, Practical Simulation in Urology
View all citing articles on Scopus
View full text