Assessing Person Fit With the Information Matrix Test
Abstract
In this manuscript, a new approach to the analysis of person fit is presented that is based on the information matrix test of White (1982). This test can be interpreted as a test of trait stability during the measurement situation. The test follows approximately a χ2-distribution. In small samples, the approximation can be improved by a higher-order expansion. The performance of the test is explored in a simulation study. This simulation study suggests that the test adheres to the nominal Type-I error rate well, although it tends to be conservative in very short scales. The power of the test is compared to the power of four alternative tests of person fit. This comparison corroborates that the power of the information matrix test is similar to the power of the alternative tests. Advantages and areas of application of the information matrix test are discussed.
References
(1998). Bayesian identification of outliers in computerized adaptive tests. Journal of the American Statistical Association, 93, 910–919.
(1991). Asymptotic expansion of the information matrix test statistic. Econometrics, 59, 787–815.
(2011). On the usefulness of a multilevel logistic regression approach to person-fit analysis. Multivariate Behavioral Research, 46, 365–388.
(1997). On the corrections to information matrix tests. Econometric Reviews, 16, 39–53.
(1987). Detecting inappropriate test scores with optimal and practical appropriateness indices. Applied Psychological Measurement, 11, 59–79.
(1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67–86.
(2004). Testing hypotheses about the person-response function in person-fit analysis. Multivariate Behavioral Research, 39, 1–35.
(2005). Global, local and graphical person-fit analysis using person response functions. Psychological Methods, 10, 101–119.
(2007). A Pearson-Type-VII item response model for assessing person fluctuations. Psychometrika, 72, 25–41.
(1994). Matrix formulae for improved score tests. Journal of Statistical Computation and Simulation, 49, 195–206.
(2007). A person fit test for IRT models for polytomous items. Psychometrika, 72, 159–180.
(2003). A Bayesian approach to person fit analysis in item response theory models. Applied Psychological Measurement, 27, 217–233.
(1981). Analysis of item response patterns: Questionable test data and dissimilar curriculum practices. Journal of Educational Measurement, 18, 133–143.
(1985). An asymptotic expansion for the null distribution of the efficient score statistic. Biometrika, 72, 653–659.
(1990). An approximately standardized person test for assessing consistency with a latent trait model. British Journal of Mathematical and Statistical Psychology, 43, 193–206.
(1994). The number of Guttman errors as a simple and powerful person-fit-statistic. Applied Psychological Measurement, 18, 311–314.
(1996). Person-fit research: An introduction. Applied Measurement in Education, 9, 3–8.
(2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107–135.
(2012). The use of the lz and lz* person-fit statistics and problems derived from model misspecification. Journal of Educational and Behavioral Statistics, 37, 758–766.
(1990). The many null distributions of person fit indices. Psychometrika, 55, 75–106.
. (2009). R: A language and environment for statistical computing [Computer software manual]. Vienna, Austria: R Development Core Team. Retrieved from www.R-project.org (ISBN 3-900051-07-0)
(2000). Using multilevel logistic regression to evaluate person-fit in IRT models. Multivariate Behavioral Research, 35, 543–568.
(2013). A systematic review of the methodology for person fit research in item response theory: Lessons about generalizability of inferences from the design of simulation studies. Psychological Test and Assessment Modeling, 55, 3–38.
(1975). The construction and interpretation of S-P tables. Tokyo, Japan: Meiji Tokyo.
(1986). A coefficient of deviant response patterns. Kwantitative Methoden, 7, 131–145.
(1992). A method for investigating the intersection of item response functions in Mokken’s non-parametric IRT model. Applied Psychological Measurement, 16, 149–157.
(2001). Asymptotic distribution of person-fit statistics with estimated person parameter. Psychometrika, 66, 331–342.
(1984). Caution indices based on item response theory. Psychometrika, 49, 95–110.
(2012). A CUSUM to detect person misfit: A discussion and some alternatives for existing procedures. Applied Psychological Measurement, 37, 420–442.
(1982). Deviant response patterns and comparability of test scores. Journal of Cross-Cultural Psychology, 13, 267–298.
(2001). CUSUM-based person-fit statistics for adaptive testing. Journal of Educational and Behavioral Statistics, 26, 199–218.
(1982). Maximum likelihood estimation of misspecified models. Econometrica, 50, 1–25.
(1980). Afterward. In G. RaschEd., Probabilistic models for some intelligence and attainment tests: With foreword and afterward by Benjamin D. Wright. Chicago, IL: Mesa Press.
(1979). Best test design. Chicago, IL: Mesa Press.
