Introduction
For the development of optimal clinical performance, the importance of linking feedback with deliberate practice is generally acknowledged [
1‐
3]. Regulatory authorities have called for feedback to feature strongly in medical training [
4,
5]. Medical students demand more feedback after assessment [
6,
7] and much advice is available for faculty on feedback delivery [
8]. It is therefore surprising that learners sometimes neglect opportunities for feedback after assessment. In one study, 50 % of students failed to access feedback on an essay examination [
9], and in another, students just achieving minimal competence in a summative Objective Structured Clinical Examination (OSCE) made least use of feedback [
10].
Feedback may not reach its full potential in practice for several reasons. Faculty find it a complex process and fear being perceived as unkind to learners, as they struggle with conflicting aims of improving learners’ future performance and building their confidence. [
11] Learners fear challenges to their own self-assessments, [
12] often wanting feedback to boost their confidence, not correct deficiencies.[
13] Feedback not aligned with a learner’s perception may be ignored [
14,
15]. Medicine’s learning culture may also limit receptivity. In studies of music, athletics and teacher education, formative critical feedback was expected to push students towards excellence, whereas this was less expected within medicine [
16‐
19].
To maximize learning from feedback, factors influencing its uptake in both high-stakes (summative) and low-stakes assessments should be explored. The summative assessment context has been shown to influence behaviour, emotions and cognitions fostering a reductionist approach which aims mainly to avoid failure and focuses attention on failing students who were ‘punished’ by resitting assessments [
20]. In contrast, students who passed, even minimally, were ignored by faculty and felt little incentive to address their weaknesses.
Some medical schools have shifted towards ‘assessment for learning’ and programmatic assessment with a focus on multiple low-stakes assessments combined with rich, narrative-based feedback throughout the period of study [
21‐
23]. However, implementing this approach in practice may not be straightforward. Recent studies have demonstrated that students may still regard the low-stakes assessments as summative hurdles to be overcome instead of learning opportunities [
24,
25].It is therefore unclear how well these modifications to the assessment system promote learning from feedback.
Within these emerging differences in institutional assessment cultures, the impact on students’ reception to feedback appears complex. We developed a study to gain further insight. Our research question was: ‘what are the factors within medical schools’ assessment systems which aid or hinder student receptivity to feedback?’ We aimed to answer the question by exploring the experiences of students from three medical schools with different approaches to assessment and feedback.
Results
Four principal themes emerged: (1) Personal agency; (2) Authenticity and relevance of assessment; (3) Grades and comparative ranking; (4) Scaffolding of feedback. They are described below, with illustrative participant quotes to illustrate the themes.
Personal agency
Factors within the assessment systems across all schools promoted, or hindered, students’ personal agency, their capacity to act and make choices using their personal abilities within the constraints and possibilities of their context. Agency was promoted by providing choice, either within compulsory assessments or by providing optional assessments, which enabled students to demonstrate knowledge acquisition to themselves and their tutors. In contrast, compulsory multiple-choice question papers reduced personal agency by preventing students from fully demonstrating their knowledge; the questions were perceived to focus on specific ‘random’ facts.
I’m often frustrated with the questions because I think, ‘Well, I’m actually good in this field but this one particular question, I don’t know.’ (School C, Focus Group (FG)1)
Students discounted much of the feedback as irrelevant for future learning or assessments. Feedback methods enabling students to act independently were more conducive to supporting personal agency than those restricting autonomy. For example, online questions and feedback, combined with the ability to revisit these at any time, were preferred to didactic tutor-led feedback sessions delivered in a group setting without provision to revisit later. Providing significant autonomy within the assessment and feedback systems fostered an aspiration towards excellence:
It [the School] really lets you….truly reach and seek that excellence because it gives you that time that you can invest in whatever you deem to be important. (School A, FG1)
By contrast, an assessment system dominated by high-stakes assessments discouraged aspiration to improve:
I think sometimes exams can be limiting because you say, ‘Oh, I don’t need to know that for an exam.’ ……that stifles learning sometimes. (School B, FG1)
Students felt in control if feedback from low-stakes assessments appeared predictive of future performance in high-stakes assessments. The feeling was the opposite if low-stakes assessments were seen as too dissimilar from the ‘real thing’.
Institutional requirements for assessments (e. g. formatting rules for completing a portfolio) limited agency and provoked much frustration and risked devaluing learning. Students sometimes tried to subvert the marking criteria to maximize learning, demonstrating tension between student agency and institutional control.
Instead of looking at what they wanted I just went on a tangent and wrote. And then that for me I was like, I know I might not get a good mark. I might not even pass, but this is actually reflective. That’s going to be useful for me in the future to look back at. (School B, FG1)
Similarly, agency was threatened when institutions set criteria for selecting specific items from overall aggregated feedback to demonstrate progress. Students perceived being forced to include critical feedback, at the expense of detailed, personal and positive feedback, even if it was unhelpful, bland and generic. They felt forced to play the ‘assessment game’.
I had feedback, ‘You’re not saying enough.’ But on the other hand I got feedback, ‘But the quality of what you’re saying is very high.’ But I was always forced to make some goals from, ‘You’re not saying enough,’ so I was forced to say more. (School C, FG1)
Students recognized that autonomy should have limits. The safety of future patients necessitated that assessment
processes identify students’ weaker areas or flaws in self-assessment. Standardization of assessments was generally
accepted, even though it limited autonomy, but acceptance was much lower if it was perceived to lack immediate
relevance to their future role as doctors.
If we’re going to do a critical
appraisal of qualitative research, why not get people to find their own piece? So if you were doing paediatrics at the
time, you could have found a paediatric paper that was relevant rather than writing about bottled water because it’s
not got any clinical relevance to what we’re doing. (School B, FG1)
Authenticity and relevance of assessment
Many assessments were seen to lack authenticity or direct relevance to students’ future work as doctors. Feedback was often ignored if assessments were perceived as irrelevant. There was a belief that it was necessary to say and do certain things to pass a clinical skills assessment (such as an OSCE); these were different from what would be done in real-life clinical practice. Feedback given on wards before an OSCE was often ignored and perceived to harm the chances of passing. In contrast, an assessment system which relied only on feedback within the clinical workplace, in the absence of high-stakes OSCEs, avoided giving students mixed messages. It allowed them to concentrate on feedback they received. Recognition that learning, and developing clinical competence, was an evolving process continuing after graduation appeared particularly helpful:
It’s kind of comforting that this process [of ward-based feedback] continues as we’re residents and that we learn at a similar rate that we’re learning here in medical school. (School A, FG2)
Grades and comparative ranking
Grades and comparative student rankings within the cohort had mixed effects on receptivity to feedback. Grades reassured students they were achieving the standards required for qualification:
I think it’s like comparing my grades to what they think is necessary. So the necessary level of knowledge to be a doctor, is my knowledge lower than that? Is it higher than that? Is it about right? (School B, FG1)
Grades provided some clarity on the expected level of performance and avoided the potential for narrative
feedback to mislead:
Well, I mean, you can get a warm and fuzzy description and be four out of ten. If you get a four then you know you need to do something to get to the six that is required to make the grade. I would much rather know that I need to work harder than have a warm and fuzzy feeling. (School B, FG2)
Information comparing performance with peers was an additional stimulus to take feedback seriously:
If I got a ranking [position relative to peers] of 50 on my MCQs and a ranking of 20 on my KFPs [Key Feature Problems], then I know I need to go and work on my MCQs. (School B, FG1)
Grades following summative, end-of-year assessments gave sufficient information; more detailed feedback was seen as superfluous:
So what was on our Year 2 paper – most of it is irrelevant to what’s going to be on our third-year paper. So to give us feedback is pointless. We’ve got a grade. That’s all we need to know. (School B, FG1)
At other times within a programme, grades without narrative feedback failed to provide sufficient information or motivation to stimulate improvement. At one school, so many students were graded ‘at the level expected’ that it was felt futile to aspire towards excellence.
And for these reports, ….. I don’t really feel stimulated to do my very best to get a good grade because I know it will just be on the threshold and I’ll be fine. (School C, FG1)
The transition from a grade-based system in a previous programme to one based on narrative feedback without grades presented a challenge for students to then find ways of checking they were ‘on track’.
You don’t have that oh, I got an A. Okay, you know, I can feel good about myself. It’s just…you don’t have that external validation. (School A, FG2)
As students adjusted to this new assessment model, they became less dependent on validating labels and focussed more on preparation for practice:
You can see that this is preparing you for real life when you don’t get an A every time you go. (School A, FG2)
As the transition continued, the absence of grading or ranking forced students to take a more nuanced view of their peers’ strengths and weaknesses. This gave them more freedom in using the feedback while preserving their own self-esteem. The lack of reassurance from ‘good enough’ grades incentivized students to aim for excellence:
We don’t know what the bare minimum is. You better have a slam-dunk to make sure. (School A, FG1)
The tendency for tutors at all schools to provide vague positive feedback frustrated students; it failed to meet aspirations to improve performance.
It means very little to me to always get these ‘great job, great job, great job’ versus someone who is trying to find ways to help me get better. (School A, FG2)
It was also commonly perceived that tutors feared giving critical feedback. Students at School A described how they had actively challenged tutors for more constructively critical feedback. There was little evidence of feedback-seeking behaviour at the other sites:
They have to be prompted because they’re afraid to write something bad in our feedback, but it’s necessary. (School A, FG2)
Scaffolding of feedback
Scaffolding, as provided by mentors, played a significant role in assisting, or occasionally hindering, receptivity to feedback. Successful mentoring helped students interpret feedback while allowing them to remain in control. Agency was promoted by reducing the scaffolding as the course progressed to prepare students for a working environment reducing dependency on feedback:
My PA [mentor]… [said]… the whole point of this system is that at some point in your career, you’re not going to have people giving you monthly evaluations necessarily, ……you’re supposed to be able to self-identify this and reflect and then do that. (School A, FG2)
A student mentoring system which provided long-term supervision (over several years), and allowed mentors to see the student’s entire feedback, enabled trust to develop. Awareness of previous feedback aided interpretation of new information:
They also know all your previous feedback, so they can kind of help you contextualize it. (School A, FG2)
Organizing the support systems in this way enabled mentors to safely challenge students’ inaccurate self-assessments:
I tend to be very hard on myself to begin with, and so I would actually meet with my PA, and she goes, … this is what I’m seeing in the evidence, and this is what you’re saying and they don’t really match. (School A, FG2)
Scaffolding did not always enhance students’ agency. Mentors could become paternalistic and decide what feedback to exclude from the portfolio:
They’ve got all different opinions about what’s good. So then one counsellor comes and says, ‘No, you should change this.’ And then next you get another counsellor who’s also going to check your portfolio and then suddenly it’s all wrong and you’ve got to change it back. (School C, FG2)
Discussion
We aimed to explore factors within different medical school assessment systems affecting student receptivity to feedback and have demonstrated several important factors. Offering students choice and independence within assessment systems promoted receptivity to feedback. Assessments perceived to lack relevance and authenticity hindered openness to feedback. By contrast, assessments and feedback were valued if authentically aligned to their future work. Grades and comparative ranking marks superficially reassured students they were ‘good enough’. Not providing grades caused initial uncertainty, but later promoted more authentic recognition that both self and peers had a complex set of strengths and weaknesses which could not easily be labelled. This encouraged an aspiration to excellence. Long-term mentor relationships assisted both students’ interpretation of feedback and enabled inaccurate self-assessment to be challenged, as long as mentors avoided paternalistic attitudes which reduced students’ agency. Students ignored feedback lacking credibility or quality.
Previous studies have shown the influence of learning culture in modulating receptivity to feedback. In music and sport, a long-term close working relationship with a mentor enables critical feedback to be accepted within an atmosphere of trust [
19]. In longitudinal integrated clerkships, authentic assessment and feedback and a supportive mentoring relationship can promote learning [
31]. To our knowledge, the importance of enabling greater student choice and independence within assessment processes to foster greater receptivity to feedback has not previously been described.
Implications for medical education
In recent decades behaviourist approaches to learning, relying on passive knowledge acquisition, have moved towards constructivist approaches and active learning strategies [
32]. By contrast, the fundamental approach towards assessment has remained behaviourist. Students are rewarded for passing or punished for failing. Many institutions, including those with active learning approaches, have maintained assessments that reward superficial learning strategies, which students unsurprisingly adopt [
33,
34]. Although more constructivist principles for assessment have been requested [
35‐
37], implementation has been limited. Our study demonstrates the potential benefits of three key constructivist principles: (i) improving the authenticity of the assessment, (ii) empowering students with a more active role and (iii) gradual descaffolding to enable transformation from a performance orientation towards a learning orientation. Active involvement in assessment and feedback processes is, however, neither easily achieved nor a panacea, and the role of the authenticity of assessments and the credibility of the mentor/coach as a knowledgeable other are important factors in this transformation. There are clearly patient safety risks if students are given excessive choice within assessments too early in the curriculum and the ensuing liberty to ignore uncomfortable feedback. Nevertheless, we argue that promoting and gradually developing a culture of receptivity to feedback will ultimately benefit patient safety.
A long-term mentoring relationship appears essential for feedback provision to stimulate effective learning. It enables trust to develop in a safe environment in which learners can be challenged. Although such relationships are common in other professions, medical education often involves multiple short-term clerkships [
19,
31]. As with the transition to problem-based learning, mentors may struggle to adopt facilitative rather than paternalistic approaches [
38].
The study was not designed to directly compare the programmatic approach to assessment with the more traditional summative assessment approach. Within each school’s assessment system we identified both helpful and counterproductive elements which impacted on receptivity to feedback. The generalization of our findings therefore does not come from stable factors that are supposed to be true in all contexts; rather our findings serve to better understand the driving forces to enable adaptability of this knowledge in the design of assessment in various contexts. Designers of assessment and feedback programmes could benefit from incorporating the more helpful elements in their programmes.
Limitations
Our study has some limitations. Only the participants’ perspectives could be interpreted by the authors to
co-construct meaning which may mean that certain perspectives may be over-represented and others may be
under-represented. We do not claim to have achieved data saturation over the six focus groups. So although we think that the themes emerging from our study are important and make good sense, we are confident that there are other interesting themes that would emerge from future replications of our study. As such the combination of such studies would serve to complete the whole picture. A further support for the sensibility of our co-constructed conclusions is that they resonate with the existing feedback literature and yet allow a more focused ‘lens’ on the interactional factors between the learner and their learning context. The decision to sample different year groups was made on good grounds, but the differences between curricula may also have led to under-representation of certain perceptions, so again we want to stress that we do not assume to have been able to provide a complete picture of all themes. To triangulate students’ self-reported perceptions, it would be important in future studies to explore the perceptions of mentors and other faculty regarding the students’ receptivity to feedback.
Suggestions for further research
Our results suggest several opportunities for further investigation. While it may be clear that there is a need to gradually shift the control over the assessment process from teacher to learner it is important to better understand the factors that could slow down or accelerate this process. Future studies seeking to understand these factors would be highly informative for assessment design. As stated above it would also be important to replicate our study in a different context to establish whether important themes regarding the uptake of feedback would be detected that our study did not find. Finally, research into the conceptualizations of faculty is of paramount importance to allow for triangulation of these themes.
Conclusions
This study has sought to understand the key elements within an assessment system which influence receptivity to feedback. Whether a medical school employs a summative assessment-based structure or adopts the principles of programmatic assessment, we should strive to make the assessments as authentic as possible, consider carefully the benefits and risks of awarding grades and use long-term mentoring to enable students to be both supported and challenged by the feedback they receive. Students perceive that they benefit from a greater degree of control over assessment and feedback processes. The trend of curriculum change towards constructivism should now be mirrored in the assessment processes. Though challenging we believe this can be achieved.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (
http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.