Abstract
Reusing questions on an examination is a concern because test administrators do not want to unfairly aid examinees by exposing them to questions they have seen on previous examinations. The purpose of this study was to investigate the effect that prior exposure of questions has on the performance of repeat examinees. Two recent administrations of an examination repeated a block of 36 multiple-choice questions. Scores for 130 repeat examinees were analyzed. Examinee ability estimates on reused questions increased but estimates on non-reused questions also increased by an equal amount. Subsequent analyses compared the match between options chosen on the first and second attempts. There is a tendency to choose the same option but it does not appear to be due to examinees remembering questions. Repeat examinees do not appear to be advantaged by seeing reused questions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Boulet, J. R., McKinley, D. W., Whelan, G. P., & Hambleton, R. K. (2003). The effect of exposure on repeat candidate scores in a high-stakes standardized patient assessment. Teaching and Learning in Medicine, 15, 227–232. doi:10.1207/S15328015TLM1504_02.
Colliver, J. A., Barrows, H. S., Vu, N. V., Verhulst, S. J., Mast, T. A., & Travis, T. A. (1991). Test security in examinations that use standardized-patient cases at one medical school. Academic Medicine, 66, 279–282.
DeChamplain, A. F., MacMillan, M. K., & Margolis, M. J. (1999). Modeling the effects of security breaches on students’ performances on a large-scale standardized patient examination. Academic Medicine, 74, S49–S51.
Hertz, N. R., & Chinn, R. N. (2003). Effects of question exposure for conventional examinations in a continuous testing environment. Paper presented at the 2003 Annual Meeting of the National Council on Measurement in Education, Chicago.
Impara, J. C., & Foster, D. (2006). Question and test development strategies to minimize test fraud. In S. M. Downing & T. M. Halydyna (Eds.), Handbook of test development (pp. 91–114). Mahwah, New Jersey: Lawrence Erlbaum Associates.
Jones, P., Smith, R. W., & Talley, D. (2006). Developing test forms for small-scale achievement testing systems. In S. M. Downing & T. M. Halydyna (Eds.), Handbook of test development (pp. 487–525). Mahwah, New Jersey: Lawrence Erlbaum Associates.
O’Neill, T., Lunz, M. E., & Thiede, K. (2000). The impact of receiving the same questions on consecutive computer-adaptive test administrations. Journal of Applied Measurement, 1, 131–151.
Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181–210. doi:10.1111/j.1745-6916.2006.00012.x.
Roediger, H. L., & Marsh, E. J. (2005). The positive and negative consequences of multiple-choice testing. Journal of Experimental Psychology. Learning, Memory, and Cognition, 31, 1155–1159. doi:10.1037/0278-7393.31.5.1155.
Stocking, M. L., & Lewis, D. (2000). Methods of controlling the exposure of questions in CAT. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 163–182). Dordrecht, The Netherlands: Kluwer Academic Publishers.
Swartz, M. H., Colliver, J. A., Cohen, D. S., & Barrows, H.·S.(1994). The effect of deliberate, excessive violations of test security on a standardized-patient examination: An extended analysis. In A. I. Rothman & R. Cohen (Eds.), Proceedings of the Sixth Ottawa Conference on Medical Education, 1994, July 26–29, Toronto (pp. 280–284). Toronto: University of Toronto Press.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wood, T.J. The effect of reused questions on repeat examinees. Adv in Health Sci Educ 14, 465–473 (2009). https://doi.org/10.1007/s10459-008-9129-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-008-9129-z