Abstract
One aspect of metacognition is the monitoring of memory or comprehension measured with retrospective confidence judgments after test taking. The research questions of the present study were whether different measures for the accuracy of such confidence judgments are stable over learning time, whether they generalize over two different tests, and whether they predict the learning outcome. In order to answer these questions, a study was conducted in which university students (N = 113) learned about the basic concepts of operant conditioning for 30 min. Knowledge tests with confidence judgments presented after each item were obtained before learning, after 10 min, and at the end of the learning session. Bias and absolute bias were calculated as absolute measures of accuracy, and gamma, Pearson’s r, andd a were calculated as relative measures of accuracy. The results showed that the absolute measures were stable, but that the relative measures were not. Furthermore, absolute bias, gamma, and Pearson’s r obtained 10 min after the beginning of the learning process predicted the learning outcome. The results are discussed with regard to research on different measures of accuracy for confidence judgments.
Article PDF
Similar content being viewed by others
References
Bannert, M. (2006). Effects of reflection prompts when learning with hypermedia.Journal of Educational Computing Research,35, 359–375. doi:10.2190/94V6-R58H-3367-G388
Benjamin, A. S., &Diaz, M. (2008). Measurement of relative metamnemonic accuracy. In J. Dunlosky & R. A. Bjork (Eds.),Handbook of metamemory and memory (pp. 73–94). New York: Psychology Press.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., &Krathwohl, D. R. (1973).Taxonomie von Lernzielen im kognitiven Bereich [Taxonomy of educational objectives] (E. Füner & R. Horn, Trans., 3rd ed.). Weinheim, Germany: Beltz.
Brewer, W. F., Sampaio, C., &Barlow, M. R. (2005). Confidence and accuracy in the recall of deceptive and nondeceptive sentences.Journal of Memory & Language,52, 618–627. doi:10.1016/j.jml.2005.01.017
Byrne, B. M. (1986). Self-concept/academic achievement relations: An investigation of dimensionality, stability, and causality.Canadian Journal of Behavioural Science,18, 173–186.
Cohen, J. (1988).Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Dunlosky, J., &Hertzog, C. (1998). Training programs to improve learning in later adulthood: Helping older adults educate themselves. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.),Metacognition in educational theory and practice (pp. 249–275). Mahwah, NJ: Erlbaum.
Dunlosky, J., Hertzog, C., Kennedy, M. R. T., &Thiede, K. W. (2005). The self-monitoring approach for effective learning.Cognitive Technology,10, 4–11.
Dunlosky, J., &Lipko, A. R. (2007). Metacomprehension: A brief history and how to improve its accuracy.Current Directions in Psychological Science,16, 228–232. doi:10.1111/j.1467-8721.2007.00509.x
Dunlosky, J., Rawson, K. A., &McDonald, S. L. (2002). Influence of practice tests on the accuracy of predicting memory performance for paired associates, sentences, and text material. In T. J. Perfect & B. L. Schwartz (Eds.),Applied metacognition (pp. 68–92). Cambridge: Cambridge University Press.
Dunlosky, J., Rawson, K. A., &Middleton, E. L. (2005). What constrains the accuracy of metacomprehension judgments? Testing the transfer-appropriate-monitoring and accessibility hypotheses.Journal of Memory & Language,52, 551–565. doi:10.1016/j.jml.2005.01.011
Ehrlinger, J., &Dunning, D. (2003). How chronic self-views influence (and potentially mislead) estimates of performance.Journal of Personality & Social Psychology,84, 5–17. doi:10.1037/0022-3514.84.1.5
Glenberg, A. M., &Epstein, W. (1987). Inexpert calibration of comprehension.Memory & Cognition,15, 84–93.
Glenberg, A. M., Sanocki, T., Epstein, W., &Morris, C. (1987). Enhancing calibration of comprehension.Journal of Experimental Psychology: General,116, 119–136. doi:10.1037/0096-3445.116.2.119
Gonzalez, R., &Nelson, T. O. (1996). Measuring ordinal association in situations that contain tied scores.Psychological Bulletin,119, 159–165. doi:10.1037/0033-2909.119.1.159
Graesser, A. C., Millis, K. K., &Zwaan, R. A. (1997). Discourse comprehension.Annual Review of Psychology,48, 163–189. doi:10.1146/ annurev.psych.48.1.163
Hacker, D. J., Bol, L., &Bahbahani, K. (2008). Explaining calibration accuracy in classroom contexts: The effects of incentives, reflection, and explanatory style.Metacognition & Learning,3, 101–121. doi:10.1007/s11409-008-9021-5
Hasselhorn, M., &Hager, W. (1989). Prediction accuracy and memory performance: Correlational and experimental tests of a metamemory hypothesis.Psychological Research,51, 147–152. doi:10.1007/ BF00309310
Jonsson, A.-C., &Allwood, C. M. (2003). Stability and variability in the realism of confidence judgments over time, content domain, and gender.Personality & Individual Differences,34, 559–574. doi:10.1016/S0191-8869(02)00028-4
Kelemen, W. L., Frost, P. J., &Weaver, C. A., III (2000). Individual differences in metacognition: Evidence against a general metacognitive ability.Memory & Cognition,28, 92–107.
Kelley, C. M., &Lindsay, D. S. (1993). Remembering mistaken for knowing: Ease of retrieval as a basis for confidence in answers to general knowledge questions.Journal of Memory & Language,32, 1–24. doi:10.1006/jmla.1993.1001
Kleitman, S., &Stankov, L. (2001). Ecological and person-oriented aspects of metacognitive processes in test-taking.Applied Cognitive Psychology,15, 321–341. doi:10.1002/acp.705
Kleitman, S., &Stankov, L. (2007). Self-confidence and metacognitive processes.Learning & Individual Differences,17, 161–173. doi:10.1016/j.lindif.2007.03.004
Koriat, A. (2007). Metacognition and consciousness. In P. D. Zelazo, M. Moscovitch, & E. Thompson (Eds.),The Cambridge handbook of consciousness (pp. 289–325). Cambridge: Cambridge University Press.
Koriat, A., Nussinson, R., Bless, H., &Shaked, N. (2008). Information-based and experience-based metacognitive judgments: Evidence from subjective confidence. In J. Dunlosky & R. A. Bjork (Eds.),Handbook of metamemory and memory (pp. 117–135). New York: Psychology Press.
Lane, J., &Lane, A. [M.] (2001). Self-efficacy and academic performance.Social Behavior & Personality,29, 687–693. doi:10.2224/ sbp.2001.29.7.687
Lane, J., Lane, A. M., &Kyprianou, A. (2004). Self-efficacy, selfesteem and their impact on academic performance.Social Behavior & Personality,32, 247–256. doi:10.2224/sbp.2004.32.3.247
Leonesio, R. J., &Nelson, T. O. (1990). Do different metamemory judgments tap the same underlying aspects of memory?Journal of Experimental Psychology: Learning, Memory, & Cognition,16, 464–470. doi:10.1037/0278-7393.16.3.464
Little, T. D., Bovaird, J. A., &Slegers, D. W. (2006). Methods for the analysis of change. In D. K. Mroczek & T. D. Little (Eds.),Handbook of personality development (pp. 181–211). Mahwah, NJ: Erlbaum.
Maki, R. H. (1995). Accuracy of metacomprehension judgments for questions of varying importance levels.American Journal of Psychology,108, 327–344. doi:10.2307/1422893
Maki, R. H. (1998). Metacomprehension of text: Influence of absolute confidence level on bias and accuracy. In D. L. Medin (Ed.),The psychology of learning and motivation (Vol. 38, pp. 223–248). San Diego: Academic Press.
Maki, R. H., Jonas, D., &Kallod, M. (1994). The relationship between comprehension and metacomprehension ability.Psychonomic Bulletin & Review,1, 126–129.
Mengelkamp, C., &Bannert, M. (2009). Judgements about knowledge: Searching for factors that influence their validity.Electronic Journal of Research in Educational Psychology,7, 163–190.
Metcalfe, J. (2002). Is study time allocated selectively to a region of proximal learning?Journal of Experimental Psychology: General,131, 349–363. doi:10.1037/0096-3445.131.3.349
Moore, D., Lin-Agler, L. M., &Zabrucky, K. M. (2005). A source of metacomprehension inaccuracy.Reading Psychology,26, 251–265. doi:10.1080/02702710590962578
Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions.Psychological Bulletin,95, 109–133. doi:10.1037/0033-2909.95.1.109
Nelson, T. O. (1996). Gamma is a measure of the accuracy of predicting performance on one item relative to another item, not of the absolute performance on an individual item.Applied Cognitive Psychology,10, 257–260.
Nelson, T. O., &Dunlosky, J. (1991). When people’s judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The “delayed-JOL effect”.Psychological Science,2, 267–270. doi:10.1111/j.1467-9280.1991.tb00147.x
Nelson, T. O., &Narens, L. (1990). Metamemory: A theoretical framework and new findings. In G. H. Bower (Ed.),The psychology of learning and motivation (Vol. 26, pp. 125–173). New York: Academic Press.
Nelson, T. O., &Narens, L. (1992). Metamemory: A theoretical framework and new findings. In T. O. Nelson (Ed.),Metacognition: Core readings (pp. 117–130). Needham Heights, MA: Allyn and Bacon.
Nietfeld, J. L., Cao, L., &Osborne, J. W. (2005). Metacognitive monitoring accuracy and student performance in the postsecondary classroom.Journal of Experimental Education,74, 7–28.
Nietfeld, J. L., Cao, L., &Osborne, J. W. (2006). The effect of distributed monitoring exercises and feedback on performance, monitoring accuracy, and self-efficacy.Metacognition & Learning,1, 159–179.
Nietfeld, J. L., &Schraw, G. (2002). The effect of knowledge and strategy training on monitoring accuracy.Journal of Educational Research,95, 131–142.
Pallier, G., Wilkinson, R., Danthiir, V., Kleitman, S., Knezevic, G., Stankov, L., &Roberts, R. D. (2002). The role of individual differences in the accuracy of confidence judgments.Journal of General Psychology,129, 257–299.
Pressley, M., &Ghatala, E. S. (1988). Delusions about performance on multiple-choice comprehension tests.Reading Research Quarterly,23, 454–464. doi:10.2307/747643
Rost, D. H. (2005).Interpretation und Bewertung pädagogisch-psychologischer Studien. Eine Einführung [Interpretation and evaluation of studies in educational psychology. An introduction]. Weinheim, Germany: Beltz.
Rost, J. (2004).Lehrbuch Testtheorie—Testkonstruktion [Test theory and test construction textbook]. Bern, Switzerland: Huber.
Sampaio, C., &Brewer, W. F. (2009). The role of unconscious memory errors in judgments of confidence for sentence recognition.Memory & Cognition,37, 158–163. doi:10.3758/MC.37.2.158
Schilling, O. (2005). Strukturgleichungsmodelle zur Analyse von Simplex-Strukturen [Structural equation models for the analysis of simplex structures]. In J. Werner (Ed.),Zeitreihenanalysen mit Beispielen aus der Psychologie (pp. 37–72). Berlin: Logos.
Schraw, G. (2009). A conceptual analysis of five measures of metacognitive monitoring.Metacognition & Learning,4, 33–45. doi:10.1007/ s11409-008-9031-3
Schraw, G., Dunkle, M. E., Bendixen, L. D., &DeBacker Roedel, T. (1995). Does a general monitoring skill exist?Journal of Educational Psychology,87, 433–444. doi:10.1037/0022-0663.87.3.433
Schraw, G., &Nietfeld, J. [L.] (1998). A further test of the general monitoring skill hypothesis.Journal of Educational Psychology,90, 236–248. doi:10.1037/0022-0663.90.2.236
Son, L. K., &Metcalfe, J. (2000). Metacognitive and control strategies in study-time allocation.Journal of Experimental Psychology: Learning, Memory, & Cognition,26, 204–221. doi:10.1037/0278-7393.26.1.204
Stankov, L. (1998). Calibration curves, scatterplots and the distinction between general knowledge and perceptual tasks.Learning & Individual Differences,10, 29–50. doi:10.1016/S1041-6080(99)80141-1
Thiede, K. W., Anderson, M. C. M., &Therriault, D. (2003). Accuracy of metacognitive monitoring affects learning of texts.Journal of Educational Psychology,95, 66–73. doi:10.1037/0022-0663.95.1.66
Thiede, K. W., &Dunlosky, J. (1999). Toward a general model of self-regulated study: An analysis of selection of items for study and self-paced study time.Journal of Experimental Psychology: Learning, Memory, & Cognition,25, 1024–1037. doi:10.1037/0278-7393.25.4.1024
Thompson, W. B., &Mason, S. E. (1996). Instability of individual differences in the association between confidence judgments and memory performance.Memory & Cognition,24, 226–234.
Tobias, S., &Everson, H. (2000). Assessing metacognitive knowledge monitoring. In G. Schraw & J. C. Impara (Eds.),Issues in the measurement of metacognition (pp. 147–222). Lincoln, NE: Buros Institute of Mental Measurements.
Weaver, C. A., III (1990). Constraining factors in calibration of comprehension.Journal of Experimental Psychology: Learning, Memory, & Cognition,16, 214–222. doi:10.1037/0278-7393.16.2.214
Weaver, C. A., III, &Kelemen, W. L. (1997). Judgments of learning at delays: Shifts in response patterns or increased metamemory accuracy?Psychological Science,8, 318–321. doi:10.1111/j.1467-9280.1997.tb00445.x
Yates, J. F. (1990).Judgment and decision making. Englewood Cliffs, NJ: Prentice Hall.
Author information
Authors and Affiliations
Corresponding author
Additional information
The present article was mainly written during a research stay of M.B. at the CoCo Research Centre, University of Sydney, Australia, which was supported by funds from the German Science Foundation (DFG: BA 2044/5-1).
Rights and permissions
About this article
Cite this article
Mengelkamp, C., Bannert, M. Accuracy of confidence judgments: Stability and generality in the learning process and predictive validity for learning outcome. Memory & Cognition 38, 441–451 (2010). https://doi.org/10.3758/MC.38.4.441
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/MC.38.4.441