Top

Quality of Life Research

Gepubliceerd in:

11-11-2023

A random item effects generalized partial credit model with a multiple imputation-based scoring procedure

Auteurs: Sijia Huang, Seungwon Chung, Li Cai

Gepubliceerd in: Quality of Life Research | Uitgave 3/2024

Abstract

Purpose

Random item effects item response theory (IRT) models have received much attention for more than a decade. However, more research is needed on random item effects IRT models for polytomous data. Additionally, to improve the utility of this new class of IRT models, the scoring issue must be addressed.

Methods

We proposed a new random item effects generalized partial credit model (GPCM), which considers both random person and random item and category-specific effects. In addition, we introduced a multiple imputation (MI)-based scoring procedure that applies to various random item effects IRT models. To evaluate the proposed model and scoring procedure, we analyzed data from a Quality of Life (QoL) scale for the Chronically Mentally III and conducted a preliminary simulation study.

Results

In the empirical data analysis, we found that patient scores generated based on the proposed model and scoring procedure were almost identical to those obtained through the conventional GPCM and scoring method. However, the standard errors (SEs) associated with the scores were slightly larger when the proposed approach was utilized. In the simulation study, we observed adequate recovery of the model parameters and patient scores.

Conclusion

The proposed model and MI-based scoring procedure contribute to the literature. The proposed model substantially reduces the number of free parameters in comparison to a conventional GPCM, which can be desired when sample sizes are small, e.g., special populations. In addition, the MI-based scoring procedure addresses the scoring issue and can be easily extended for scoring with other random item effects IRT models.

vorige artikel Using a multi-stakeholder co-design process to develop a health service organisation-wide patient reported outcome measure collection system

volgende artikel Physical and occupational therapists' perceptions of sustainability of a knowledge translation intervention to improve the use of outcome measures in inpatient rehabilitation: a qualitative study

Alleen toegankelijk voor geautoriseerde gebruikers

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison-Wesley.

Cai, L., Choi, K., Hansen, M., & Harrell, L. (2016). Item response theory. Annual Review of Statistics and Its Application, 3(1), 297–321. https://doi.org/10.1146/annurev-statistics-041715-033702CrossRefADS

Huang, S., & Cai, L. (2023). Multidimensional item response theory. In R. Tierney, F. Rizvi, & K. Ercikan (Eds.), International Encyclopedia of Education (4th ed., pp. 86–98). Elsevier. https://doi.org/10.1016/B978-0-12-818630-5.10011-9CrossRef

Thissen, D., & Steinberg, L. (2009). Item response theory. In R. Millsap & A. Maydeu-Olivares (Eds.), The Sage handbook of quantitative methods in psychology (pp. 148–177). Sage.CrossRef

Reckase, M. D. (2009). Multidimensional item response theory. Springer. https://doi.org/10.1007/978-0-387-89976-3CrossRef

Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159–176. https://doi.org/10.1177/014662169201600206CrossRef

Muraki, E. (1997). A generalized partial credit model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 153–164). Springer. https://doi.org/10.1007/978-1-4757-2691-6_9CrossRef

Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 395–479). Addison-Wesley.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Routledge. https://doi.org/10.4324/9780203056615CrossRef

10.

Andersen, E. B. (1970). Asymptotic properties of conditional maximum-likelihood estimators. Journal of the Royal Statistical Society Series B (Methodological), 32(2), 283–301. https://doi.org/10.1111/j.2517-6161.1970.tb00842.xMathSciNetCrossRef

11.

Ghosh, M. (1995). Inconsistent maximum likelihood estimators for the Rasch model. Statistics and Probability Letters, 23(2), 165–170. https://doi.org/10.1016/0167-7152(94)00109-LMathSciNetCrossRef

12.

Neyman, J., & Scott, E. L. (1948). Consistent estimates based on partially consistent observations. Econometrica, 16(1), 1–32. https://doi.org/10.2307/1914288MathSciNetCrossRef

13.

Meng, X., & Schilling, S. (1996). Fitting full-information item factor models and an empirical investigation of bridge sampling. Journal of the American Statistical Association, 91(435), 1254–1267. https://doi.org/10.1080/01621459.1996.10476995CrossRef

14.

Wei, G. C., & Tanner, M. A. (1990). A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms. Journal of the American Statistical Association, 85(411), 699–704. https://doi.org/10.2307/2290005CrossRef

15.

Cai, L. (2010). Metropolis-Hastings Robbins-Monro algorithm for confirmatory item factor analysis. Journal of Educational and Behavioral Statistics, 35(3), 307–335. https://doi.org/10.3102/1076998609353115CrossRef

16.

Cai, L. (2010). High-dimensional exploratory item factor analysis by a Metropolis-Hastings Robbins–Monro algorithm. Psychometrika, 75(1), 33–57. https://doi.org/10.1007/s11336-009-9136-xMathSciNetCrossRef

17.

De Boeck, P. (2008). Random Item IRT models. Psychometrika, 73(4), 533–559. https://doi.org/10.1007/s11336-008-9092-xMathSciNetCrossRef

18.

Janssen, R., Tuerlinckx, F., Meulders, M., & De Boeck, P. (2000). A hierarchical IRT model for criterion-referenced measurement. Journal of Educational and Behavioral Statistics, 25(3), 285–306. https://doi.org/10.3102/10769986025003285CrossRef

19.

Van den Noortgate, W., De Boeck, P., & Meulders, M. (2003). Cross-classification multilevel logistic models in psychometrics. Journal of Educational and Behavioral Statistics, 28(4), 369–386. https://doi.org/10.3102/10769986028004369CrossRef

20.

de Jong, M. G., & Steenkamp, J.-B.E.M. (2010). Finite mixture multilevel multidimensional ordinal IRT models for large scale cross-cultural research. Psychometrika, 75(1), 3–32. https://doi.org/10.1007/s11336-009-9134-zMathSciNetCrossRef

21.

de Jong, M. G., Steenkamp, J.-B.E.M., & Fox, J.-P. (2007). Relaxing measurement invariance in cross-national consumer research using a hierarchical IRT model. Journal of Consumer Research, 34(2), 260–278. https://doi.org/10.1086/518532CrossRef

22.

Fox, J.-P., & Verhagen, A. J. (2010). Random item effects modeling for cross-national survey data. In D. P. Schmidt & J. Billie (Eds.), Cross-cultural analysis: Methods and applications (pp. 467–488). Routledge. https://doi.org/10.4324/9781315537078CrossRef

23.

Rijmen, F., & Jeon, M. (2013). Fitting an item response theory model with random item effects across groups by a variational approximation method. Annals of Operations Research, 206(1), 647–662. https://doi.org/10.1007/s10479-012-1181-7MathSciNetCrossRef

24.

Geerlings, H., Glas, C. A. W., & van der Linden, W. J. (2011). Modeling rule-based item generation. Psychometrika, 76(2), 337–359. https://doi.org/10.1007/s11336-011-9204-xMathSciNetCrossRef

25.

Glas, C. A. W., & van der Linden, W. J. (2003). Computerized adaptive testing with item cloning. Applied Psychological Measurement, 27(4), 247–261. https://doi.org/10.1177/0146621603027004001MathSciNetCrossRef

26.

Cho, S. J., & Rabe-Hesketh, S. (2011). Alternating imputation posterior estimation of models with crossed random effects. Computational Statistics and Data Analysis, 55(1), 12–25. https://doi.org/10.1016/j.csda.2010.04.015MathSciNetCrossRef

27.

Johnson, M. S., & Sinharay, S. (2005). Calibration of polytomous item families using Bayesian hierarchical modeling. Applied Psychological Measurement, 29(5), 369–400. https://doi.org/10.1177/0146621605276675MathSciNetCrossRef

28.

Wang, W.-C., & Qiu, X.-L. (2013). A multidimensional and multilevel extension of a random-effect approach to subjective judgment in rating scales. Multivariate Behavioral Research, 48(3), 398–427. https://doi.org/10.1080/00273171.2013.784861CrossRefPubMed

29.

Wang, W.-C., Wilson, M., & Shih, C.-L. (2006). Modeling randomness in judging rating scales with a random-effects rating scale model. Journal of Educational Measurement, 43(4), 335–353. https://doi.org/10.1111/j.1745-3984.2006.00020.xCrossRef

30.

Wang, W.-C., & Wu, S.-L. (2011). The random-effect generalized rating scale model. Journal of Educational Measurement, 48(4), 441–456.CrossRef

31.

Huang, S., Luo, J., & Cai, L. (2022). An explanatory multidimensional random item effects rating scale model. Educational and Psychological Measurement. https://doi.org/10.1177/00131644221140906CrossRefPubMedPubMedCentral

32.

Cai, L. (2015). Lord-Wingersky algorithm version 2.0 for hierarchical item factor models with applications in test scoring, scale alignment, and model fit testing. Psychometrika, 80(2), 535–559. https://doi.org/10.1007/s11336-014-9411-3MathSciNetCrossRefPubMed

33.

Huang, S., & Cai, L. (2021). Lord-Wingersky algorithm version 2.5 with applications. Psychometrika, 86(4), 973–993. https://doi.org/10.1007/s11336-021-09785-yMathSciNetCrossRefPubMedPubMedCentral

34.

Lord, F. M., & Wingersky, M. S. (1984). Comparison of IRT true-score and equipercentile observed-score “equatings.” Applied Psychological Measurement, 8(4), 453–461. https://doi.org/10.1177/014662168400800409CrossRef

35.

Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37(1), 29–51. https://doi.org/10.1007/BF02291411CrossRef

36.

Thissen, D., & Cai, L. (2016). Nominal categories models. Handbook of item response theory (pp. 79–102). Chapman and Hall/CRC.

37.

Thissen, D., Cai, L., & Bock, R. D. (2010). The nominal categories item response model. Handbook of polytomous item response theory models (pp. 43–75). Routledge/Taylor & Francis Group.

38.

Huang, S. (2021). Estimation of Cross-Classified Multilevel Item Response Theory Models with Metropolis-Hastings Robbins-Monro Algorithm (Publication Number 28547010) [Ph.D., University of California, Los Angeles]. ProQuest Dissertations & Theses Global.

39.

Rubin, D. B. (1976). Inference and missing data. Biometrika, 63(3), 581–592. https://doi.org/10.1093/biomet/63.3.581MathSciNetCrossRef

40.

Huang, S., & Cai, L. (2019). Improving standard error estimates in multistage estimation: A multiple imputation (MI) based approach. Multivariate Behavioral Research, 54(1), 154–154. https://doi.org/10.1080/00273171.2018.1557034CrossRefPubMed

41.

Yang, J. S., Hansen, M., & Cai, L. (2012). Characterizing sources of uncertainty in IRT scale scores. Educational and Psychological Measurement, 72(2), 264–290. https://doi.org/10.1177/0013164411410056CrossRefPubMed

42.

Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51(4), 567–577. https://doi.org/10.1007/BF02295596CrossRef

43.

Andrich, D. (1978). Application of a psychometric rating model to ordered categories which are scored with successive integers. Applied Psychological Measurement, 2(4), 581–594. https://doi.org/10.1177/014662167800200413CrossRef

44.

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561–573. https://doi.org/10.1007/BF02293814CrossRef

45.

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. https://doi.org/10.1007/BF02296272CrossRef

46.

Huang, S., & Cai, L. (2023). Cross-classified item response theory modeling with an application to student evaluation of teaching. Journal of Educational and Behavioral Statistics. https://doi.org/10.3102/10769986231193351CrossRef

47.

Chung, S., & Cai, L. (2021). Cross-classified random effects modeling for moderated item calibration. Journal of Educational and Behavioral Statistics, 46(6), 651–681. https://doi.org/10.3102/1076998620983908CrossRef

48.

Falk, C. F., & Cai, L. (2016). A flexible full-information approach to the modeling of response styles. Psychological Methods, 21(3), 328–347. https://doi.org/10.1037/met0000059CrossRefPubMed

49.

Monroe, S., & Cai, L. (2014). Estimation of a Ramsay-curve item response theory model by the Metropolis-Hastings Robbins–Monro algorithm. Educational and Psychological Measurement, 74(2), 343–369. https://doi.org/10.1177/0013164413499344CrossRef

50.

Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Sage Publications Inc.

51.

Enders, C. K. (2010). Applied missing data analysis. Guilford Press.

52.

Lehman, A. F. (1988). A Quality of Life Interview for the chronically mentally ill. Evaluation and Program Planning, 11(1), 51–62. https://doi.org/10.1016/0149-7189(88)90033-XCrossRef

53.

Cai, L. (2020). flexMIRT version 3.62: Flexible Multilevel Multidimensional Item Analysis and Test Scoring.

54.

Chung, S., & Cai, L. (2019). Alternative multiple imputation inference for categorical structural equation modeling. Multivariate Behavioral Research, 54(3), 323–337. https://doi.org/10.1080/00273171.2018.1523000CrossRefPubMed

Titel: A random item effects generalized partial credit model with a multiple imputation-based scoring procedure
Auteurs: Sijia Huang
Seungwon Chung
Li Cai
Publicatiedatum: 11-11-2023
Uitgeverij: Springer International Publishing
Gepubliceerd in: Quality of Life Research / Uitgave 3/2024
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI: https://doi.org/10.1007/s11136-023-03551-6

Bohn Stafleu van Loghum

Deel dit onderdeel of sectie (kopieer de link)

Abstract

Purpose

Methods

Results

Conclusion

Log in om toegang te krijgen

Andere artikelen Uitgave 3/2024

Sexual quality of life after gynaecological cancer: what young women want

Using a multi-stakeholder co-design process to develop a health service organisation-wide patient reported outcome measure collection system

Development of the PROMIS pediatric stigma and extension to the PROMIS pediatric stigma: skin item banks

Using parameter perturbation to facilitate transparency in measure development

QALY-type preference and willingness-to-pay among end-of-life patients with cancer treatments: a pilot study using discrete choice experiment

Mapping IWQOL-Lite onto EQ-5D-5L and SF-6Dv2 among overweight and obese population in China