What this paper adds
Introduction
Methods
Design
Setting
Phase 1: Qualitative
Phase 2: Quantitative
Results
Phase 1: Qualitative
I’ve actually found it really helpful to have time one on one to talk about academic things and SMART goals because I feel like sometimes the feedback from tests is super generic. You either met expectations, you were borderline, or you didn’t, and I often don’t really know what does that mean for what I’m doing and what I need to change? (Group 3)
I think coaching time is to just talk about what you want. People in our sessions have talked about very personal issues, people have talked about the curriculum, people have talked about SMART goals …just that openness … that’s a really helpful setting. (Group 2)
It’s not helpful to my learning or my long-term career goals because often what I’m doing is taking things that I’m already doing and then somehow squishing them into this format that I can just put on this dashboard. (Group 1)
I don’t always go into meetings with my coach with clear hopes or goals, but actually for my last session I had no ideas, and she was really helpful in talking to me and figuring out where to improve.’ (Group 3)
Phase 2: Quantitative
Factors (α coefficient) | Item | Meana | Standard deviation (SD) | Factor mean (SD) |
---|---|---|---|---|
Factor 1: SMARTb goals (α = 0.88) | I regularly set SMART goals for myself to improve my learning | 2.82 | 0.99 | 3.25 (0.91) |
SMART goals are a useful tool | 3.36 | 1.02 | ||
I find it useful to create SMART goals related to my academic performance | 3.01 | 1.15 | ||
I find it useful to create SMART goals related to my career planning | 3.49 | 1.15 | ||
I find it useful to create SMART goals related to my well-being | 3.75 | 1.01 | ||
I keep track of the progress of my SMART goals | 2.97 | 1.02 | ||
Factor 2: Dashboard usage (α = 0.82) | I usually choose which goal to create based on data on my performance from the dashboard | 2.45 | 1.14 | 3.36 (0.64) |
I am able to review my individual performance readily using the dashboard | 3.67 | 0.94 | ||
It is helpful to review my performance compared with other students using the dashboard | 3.71 | 0.97 | ||
The breadth of data in the dashboard is sufficient for me to assess my academic performance | 3.32 | 0.94 | ||
The exam grade reports in the dashboard help in assessing my performance | 3.55 | 0.90 | ||
I find it easy to track my exam scores over time to monitor improvement in my performance | 3.25 | 0.96 | ||
Reviewing my dashboard motivates me to improve my performance | 3.54 | 0.87 | ||
The information about my performance in the dashboard is accurate | 3.70 | 0.78 | ||
The information about my performance in the dashboard reflects my progression as a medical student | 3.40 | 0.82 | ||
Factor 3: Learning strategy (α = 0.81) | I use effective learning tools and strategies | 3.91 | 0.68 | 3.72 (0.64) |
When I study, I test myself to check my understanding of what I’ve studied | 3.94 | 0.86 | ||
I know how to identify the best learning resources to make progress in an area that I need to improve in academically | 3.46 | 0.91 | ||
I have the skills required to be a lifelong learner | 4.08 | 0.75 | ||
I am adequately supported to implement my SMART learning goals | 3.68 | 0.73 | ||
I currently know how to interpret feedback and data/scores to improve my academic performance | 3.48 | 0.81 | ||
Factor 4: Coaching for improvement (α = 0.71) | My coach helps me create or adjust my SMART goals | 3.83 | 0.88 | 3.67 (0.79) |
Working with my coach helps me understand how to use performance data and feedback to create SMART learning goals | 3.52 | 1.02 | ||
My coach helps me to assess and reflect on the progress I have made on my goals | 3.80 | 0.84 | ||
Factor 5: Reflection (α = 0.70) | When I receive feedback, I often reflect and set new goals | 3.79 | 0.81 | 3.68 (0.64) |
I direct my learning based on my individual learning needs | 4.19 | 0.72 | ||
I mainly use sources other than the dashboard to review how I am doing in medical school | 2.94 | 0.97 | ||
If I’m not on track with my goals, I try to make adjustments and find resources to achieve those goals on my own | 3.92 | 0.80 | ||
When I receive feedback, I use that information to change my behaviour | 4.18 | 0.59 |