Swipe om te navigeren naar een ander artikel
Students learn more effectively when they know what they do not know. Gaining insight into students’ metacognitive awareness is needed as misalignment between actual and self-perceived knowledge impedes their learning process. The optimal method of measuring self-perceived knowledge is still under debate. In this study, we evaluate the use of psychology-derived post-decision wagering for mapping students self-perceived knowledge.
Students (n = 71) performed a pre-test on medical physiology, followed by a teacher-moderated discussion and a post-test with isomorph questions. Half of the students rated their self-perceived knowledge on each question using post-decision wagering, i. e. betting 1–5 points on the correctness of their answer, whereas the other half used a 5-point Likert scale to rate their confidence.
Self-perceived knowledge scores were higher for post-decision wagering (pre: 3.75 ± 0.14, post: 4.60 ± 0.07) compared with Likert scales (pre: 3.13 ± 0.08, post: 3.92 ± 0.08) despite similar actual knowledge scores. Furthermore, Likert ratings showed a near-normal distribution, whereas wagers were placed preferentially using the outer ends of the scale. Correlations between mean actual and self-perceived knowledge scores were low in both groups. On average, 8.5% of responses were classified as misconceptions, defined as highly confident incorrect answers.
Despite the presumed reliability of post-decision wagering, our findings suggest that we should adhere to the use of Likert scales as a balanced measure for self-perceived knowledge in medical education. Moreover, the prevalence of misconceptions did not alter after instruction, indicating a need for instructional designs that enhance students’ conceptual understanding in basic sciences.
Ziegler B, Montplaisir L. Student Perceived and Determined Knowledge of Biology Concepts in an Upper-Level Biology Course. CBE Life Sci Educ. IEEE Trans Med Imaging. 2014;13:322:30.
Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77:1121–34. CrossRef
Flavell JH. Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. Am Psychol. 1979;34:906. CrossRef
Zohar A, Barzilai S. A review of research on metacognition in science education: Current and future directions. Stud Sci Educ. IEEE Trans Med Imaging. 2013;49:121:69.
Pintrich PR. The role of metacognitive knowledge in learning, teaching, and assessing. Theory Pract. 2002;41:219–25. CrossRef
Cavers M, Ling J. Confidence Weighting Procedures for Multiple-Choice Tests. In: Chen DG, Chen J, Lu X, Yi G, Yu H, editors. Advanced Statistical Methods in Data Science. ICSA Book Series in Statistics. Singapore: Springer; 2016.
Sparck EM, Bjork EL, Bjork RA. On the learning benefits of confidence-weighted testing. Cogn Res Princ Implic. IEEE Trans Med Imaging. 2016;1:3.
Swartz SM. Acceptance and Accuracy of Multiple Choice, Confidence-Level, and Essay Question Formats for Graduate Students. J Educ Bus. 2006;81:215–20. CrossRef
Hopkins KD, Hakstian AR, Hopkins BR. Validity and reliability consequences of confidence weighting. Educ Psychol Meas. 1973;33:135–41. CrossRef
Bokhorst FD. Confidence-weighting and the validity of achievement tests. Psych Rep. IEEE Trans Med Imaging. 1986;59:383:6.
Brinkman DJ, Tichelaar J, van Agtmael MA, de Vries TP, Richir MC. Self-reported confidence in prescribing skills correlates poorly with assessed competence in fourth-year medical students. J Clin Pharmacol. 2015;55:825–30. CrossRef
Kampmeyer D, Matthes J, Herzig S. Lucky guess or knowledge: A cross-sectional study using the Bland and Altman analysis to compare confidence-based testing of pharmacological knowledge in 3rd and 5th year medical students. Adv Health Sci Educ. IEEE Trans Med Imaging. 2015;20:431:40.
Rangel RH, Möller L, Sitter H, Stibane T, Strzelczyk A. Sure, or unsure? Measuring students’ confidence and the potential impact on patient safety in multiple-choice questions. Med Teach. 2017;39:1189–94. CrossRef
Curtis DA, Lind SL, Boscardin CK, Dellinges M. Does student confidence on multiple-choice question assessments provide useful information? Med Educ. 2013;47:578–84. CrossRef
Lai NM, Teng CL. Self-perceived competence correlates poorly with objectively measured competence in evidence based medicine among medical students. Bmc Med Educ. 2011;11:25. CrossRef
Brydges R, Butler D. A reflective analysis of medical education research on self-regulation in learning and practice. Med Educ. 2012;46:71–9. CrossRef
Sandars J. The use of reflection in medical education: AMEE Guide. Med Teach. 2009;31(44):685–95. CrossRef
Leggett H, Sandars J, Burns P. Helping students to improve their academic performance: a pilot study of a workbook with self-monitoring exercises. Med Teach. 2012;34:751–3. CrossRef
Tanner K, Allen D. Approaches to biology teaching and learning: understanding the wrong answers—teaching toward conceptual change. CBE Life Sci Educ. IEEE Trans Med Imaging. 2005;4:112:7.
Woods NN. Science is fundamental: the role of biomedical knowledge in clinical reasoning. Med Educ. 2007;41:1173–7. CrossRef
Dawson-Saunders B, Feltovich PJ, Coulson RL, Steward DE. A survey of medical school teachers to identify basic biomedical concepts medical students should understand. Acad Med. 1990;65:448–54. CrossRef
Michael JA, Wenderoth MP, Modell HI, et al. Undergraduates’ understanding of cardiovascular phenomena. Adv Physiol Educ. 2002;26:72–84. CrossRef
Michael J. What makes physiology hard for students to learn? Results of a faculty survey. Adv Physiol Educ. 2007;31:34–40. CrossRef
Versteeg M, Wijnen-Meijer M, Steendijk P. Informing the uninformed: a multitier approach to uncover students’ misconceptions on cardiovascular physiology. Adv Physiol Educ. 2018;43:7–14. CrossRef
Badenhorst E, Hartman N, Mamede S. How Biomedical Misconceptions May Arise and Affect Medical Students’ Learning: A Review of Theoretical Perspectives and Empirical Evidence. Health Prof Educ. IEEE Trans Med Imaging. 2016;2:10:7.
Posner GJ, Strike KA, Hewson PW, Gertzog WA. Accommodation of a scientific conception: Toward a theory of conceptual change. Sci Educ. IEEE Trans Med Imaging. 1982;66:211:27.
Hasan S, Bagayoko D, Kelley EL. Misconceptions and the certainty of response index (CRI). Phys Educ. IEEE Trans Med Imaging. 1999;34:294:9.
Gunstone RF, Mitchell IJ. Metacognition and conceptual change. In: Mintzes JJ, Wandersee JH, Novak JD, editors. Teaching science for understanding. A human constructivist view. San Diego, CA: Academic Press; 2005. pp. 133–63. CrossRef
Schraw G, Dennison RS. Assessing metacognitive awareness. Contemp Educ Psychol. 1994;19:460–75. CrossRef
Tippett CD. Refutation text in science education: A review of two decades of research. Int J Sci Math Educ. 2010;8:951–70. CrossRef
Lin LM, Zabrucky KM. Calibration of comprehension: Research and implications for education and instruction. Contemp Educ Psychol. 1998;23:345–91. CrossRef
Caleon IS, Subramaniam R. Do students know what they know and what they don’t know? Using a four-tier diagnostic test to assess the nature of students’ alternative conceptions. Res Sci Educ. IEEE Trans Med Imaging. 2010;40:313:37.
Sreenivasulu B, Subramaniam R. Exploring undergraduates’ understanding of transition metals chemistry with the use of cognitive and confidence measures. Res Sci Educ. IEEE Trans Med Imaging. 2014;44:801:28.
Yan YK, Subramaniam R. Using a multi-tier diagnostic test to explore the nature of students’ alternative conceptions on reaction kinetics. Chem Educ Res Pract. IEEE Trans Med Imaging. 2018;19:213:26.
Brandriet AR, Bretz SL. The development of the redox concept inventory as a measure of students’ symbolic and particulate redox understandings and confidence. J Chem Educ. 2014;91:1132–44. CrossRef
Koch C, Preuschoff K. Betting the house on consciousness. Nat Neurosci. 2007;10:140–1. CrossRef
Persaud N, McLeod P, Cowey A. Post-decision wagering objectively measures awareness. Nat Neurosci. 2007;10:257–61. CrossRef
Dienes Z, Scott R. Measuring unconscious knowledge: Distinguishing structural knowledge and judgment knowledge. Psychol Res. 2005;69:338–51. CrossRef
Wierzchoń M, Wronka E, Paulewicz B, Szczepanowski R. Post-decision wagering affects metacognitive awareness of emotional stimuli: an event related potential study. PLoS ONE. 2016;11:e159516. CrossRef
Sandberg K, Timmermans B, Overgaard M, Cleeremans A. Measuring consciousness: is one measure better than the other? Conscious Cogn. 2010;19:1069–78. CrossRef
Dienes Z, Seth A. Gambling on the unconscious: A comparison of wagering and confidence ratings as measures of awareness in an artificial grammar task. Conscious Cogn. 2010;19:674–81. CrossRef
Fleming SM, Dolan RJ. Effects of loss aversion on post-decision wagering: implications for measures of awareness. Conscious Cogn. 2010;19:352–63. CrossRef
Palizvan MR, Nejad MR, Jand A, Rafeie M. Cardiovascular physiology misconceptions and the potential of cardiovascular physiology teaching to alleviate these. Med Teach. 2013;35:454–8. CrossRef
Lisk K, Agur AM, Woods NN. Exploring cognitive integration of basic science and its effect on diagnostic reasoning in novices. Perspect Med Educ. 2016;5:147–53. CrossRef
Goldszmidt M, Minda JP, Devantier SL, Skye AL, Woods NN. Expanding the basic science debate: the role of physics knowledge in interpreting clinical findings. Adv Health Sci Educ. IEEE Trans Med Imaging. 2012;17:547:55.
Bell P, Volckmann D. Knowledge surveys in general chemistry: confidence, overconfidence, and performance. J Chem Educ. 2011;88:1469–76. CrossRef
Favazzo L, Willford JD, Watson RM. Correlating student knowledge and confidence using a graded knowledge survey to assess student learning in a general microbiology classroom. J Microbiol Biol Educ. 2014;15:251. CrossRef
Lindsey BA, Nagel ML. Do students know what they know? Exploring the accuracy of students’ self-assessments. Phys Rev Spec Top-Phys Educ R. IEEE Trans Med Imaging. 2015;11:20103.
Potgieter M, Malatje E, Gaigher E, Venter E. Confidence versus performance as an indicator of the presence of alternative conceptions and inadequate problem-solving skills in mechanics. Int J Sci Educ. 2010;32:1407–29. CrossRef
Ravesloot CJ, Van der Schaaf MF, Muijtjens AM, et al. The don’t know option in progress testing. Adv Health Sci Educ. IEEE Trans Med Imaging. 2015;20:1325:38.
Anderson RC, Pearson PD. A schema-theoretic view of basic processes in reading comprehension. In: Pearson PD, Barr R, Kamil ML, Mosenthal P, editors. Handbook of reading research. New York: Longman; 1984. pp. 255–91.
Bransford JD, Johnson MK. Contextual prerequisites for understanding: Some investigations of comprehension and recall. J Verbal Learn Verbal Behav. 1972;11:717–26. CrossRef
Alba JW, Hasher L. Is memory schematic? Psychol Bull. 1983;93:203. CrossRef
Tse D, Langston RF, Kakeyama M, et al. Schemas and memory consolidation. Science. 2007;316:76–82. CrossRef
van Kesteren MT, Ruiter DJ, Fernández G, Henson RN. How schema and novelty augment memory formation. Trends Neurosci. 2012;35:211–9. CrossRef
Hewson MG, Hewson PW. Effect of instruction using students’ prior knowledge and conceptual change strategies on science learning. J Res Sci Teach. 1983;20:731–43. CrossRef
Hailikari T, Katajavuori N, Lindblom-Ylanne S. The relevance of prior knowledge in learning and instructional design. Am J Pharm Educ. 2008;72:113. CrossRef
- Putting post-decision wagering to the test: a measure of self-perceived knowledge in basic sciences?
- Bohn Stafleu van Loghum