Skip to main content
main-content
Top

Tip

Swipe om te navigeren naar een ander artikel

Gepubliceerd in: Quality of Life Research 4/2009

01-05-2009

Having a fit: impact of number of items and distribution of data on traditional criteria for assessing IRT’s unidimensionality assumption

Auteurs: Karon F. Cook, Michael A. Kallen, Dagmar Amtmann

Gepubliceerd in: Quality of Life Research | Uitgave 4/2009

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Purpose

Confirmatory factor analysis fit criteria typically are used to evaluate the unidimensionality of item banks. This study explored the degree to which the values of these statistics are affected by two characteristics of item banks developed to measure health outcomes: large numbers of items and nonnormal data.

Methods

Analyses were conducted on simulated and observed data. Observed data were responses to the Patient-Reported Outcome Measurement Information System (PROMIS) Pain Impact Item Bank. Simulated data fit the graded response model and conformed to a normal distribution or mirrored the distribution of the observed data. Confirmatory factor analyses (CFA), parallel analysis, and bifactor analysis were conducted.

Results

CFA fit values were found to be sensitive to data distribution and number of items. In some instances impact of distribution and item number was quite large.

Conclusions

We concluded that using traditional cutoffs and standards for CFA fit statistics is not recommended for establishing unidimensionality of item banks. An investigative approach is favored over reliance on published criteria. We found bifactor analysis to be appealing in this regard because it allows evaluation of the relative impact of secondary dimensions. In addition to these methodological conclusions, we judged the items of the PROMIS Pain Impact bank to be sufficiently unidimensional for item response theory (IRT) modeling.
Bijlagen
Alleen toegankelijk voor geautoriseerde gebruikers
Literatuur
1.
go back to reference Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahway, NJ: Lawrence Erlbaum Associates, Publishers. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahway, NJ: Lawrence Erlbaum Associates, Publishers.
2.
go back to reference Hambleton, R., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage Publishing, Inc. Hambleton, R., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage Publishing, Inc.
6.
go back to reference Wainer, H. (1990). Computerized adaptive testing: A primer. Hillsdale, NJ: Lawrence Erlbaum Associates. Wainer, H. (1990). Computerized adaptive testing: A primer. Hillsdale, NJ: Lawrence Erlbaum Associates.
7.
8.
go back to reference McDonald, R. (1981). The dimensionality of test and items. The British Journal of Mathematical and Statistical Psychology, 34, 100–117. McDonald, R. (1981). The dimensionality of test and items. The British Journal of Mathematical and Statistical Psychology, 34, 100–117.
12.
go back to reference Bjorner, J. B., Kosinski, M., & Ware, J. E., Jr. (2003). Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the headache impact test (HIT). Quality of Life Research, 12, 913–933. doi: 10.​1023/​A:​1026163113446. PubMedCrossRef Bjorner, J. B., Kosinski, M., & Ware, J. E., Jr. (2003). Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the headache impact test (HIT). Quality of Life Research, 12, 913–933. doi: 10.​1023/​A:​1026163113446. PubMedCrossRef
14.
go back to reference Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: The Guilford Press. Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: The Guilford Press.
15.
go back to reference Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
21.
go back to reference Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–172). Newbury Park, CA: Sage Publications. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–172). Newbury Park, CA: Sage Publications.
22.
go back to reference Hu, L., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 76–79). Thousand Oaks, CA: Sage Publications. Hu, L., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 76–79). Thousand Oaks, CA: Sage Publications.
23.
go back to reference Hu, L. T., & Bentler, P. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. CrossRef Hu, L. T., & Bentler, P. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. CrossRef
24.
go back to reference Muthen, B. O., & Muthen, L. K. (2001). Mplus user’s guide. Los Angeles, CA: Muthen & Muthen. Muthen, B. O., & Muthen, L. K. (2001). Mplus user’s guide. Los Angeles, CA: Muthen & Muthen.
25.
go back to reference Yu, C. Y. (2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes. Doctoral dissertation, University of California, Los Angeles. Yu, C. Y. (2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes. Doctoral dissertation, University of California, Los Angeles.
28.
go back to reference Joreskog, K. G., & Sorbom, D. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. Lincolnwood, IL: Scientific Software International, Inc. Joreskog, K. G., & Sorbom, D. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. Lincolnwood, IL: Scientific Software International, Inc.
29.
go back to reference Bentler, P. M. (1995). EQS structural equations program manual. Encino, CA: Multivariate Software. Bentler, P. M. (1995). EQS structural equations program manual. Encino, CA: Multivariate Software.
30.
go back to reference Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. The British Journal of Mathematical and Statistical Psychology, 37, 62–83. PubMed Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. The British Journal of Mathematical and Statistical Psychology, 37, 62–83. PubMed
31.
32.
go back to reference McDonald, R. P. (1999). Test theory: A unified treatment. Mahway, NJ: Lawrence Earlbaum. McDonald, R. P. (1999). Test theory: A unified treatment. Mahway, NJ: Lawrence Earlbaum.
33.
go back to reference Kline, R. B. (1998). Principles and practice of structural equation modeling. New York, NY: The Guilford Press. Kline, R. B. (1998). Principles and practice of structural equation modeling. New York, NY: The Guilford Press.
34.
go back to reference West, S. G., Finch, J. F., & Curran, P. J. (1995). SEM with nonnormal variables. Thousand Oaks, CA: Sage Publications. West, S. G., Finch, J. F., & Curran, P. J. (1995). SEM with nonnormal variables. Thousand Oaks, CA: Sage Publications.
35.
go back to reference Joreskog, K. G. (2005). Structural equation modeling with ordinal variables using LISREL. Lincolnwood, IL: Scientific Software International, Inc. Joreskog, K. G. (2005). Structural equation modeling with ordinal variables using LISREL. Lincolnwood, IL: Scientific Software International, Inc.
36.
go back to reference Yuan, K. H., & Bentler, P. M. (1997). Mean and covariance structure analysis: Theoretical and practical improvements. Journal of the American Statistical Association, 92, 767–774. doi: 10.​2307/​2965725. CrossRef Yuan, K. H., & Bentler, P. M. (1997). Mean and covariance structure analysis: Theoretical and practical improvements. Journal of the American Statistical Association, 92, 767–774. doi: 10.​2307/​2965725. CrossRef
37.
go back to reference O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32, 396–402. O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32, 396–402.
40.
go back to reference Yung, Y. F., Thissen, D., & McLeod, L. D. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64, 113–128. doi: 10.​1007/​BF02294531. CrossRef Yung, Y. F., Thissen, D., & McLeod, L. D. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64, 113–128. doi: 10.​1007/​BF02294531. CrossRef
42.
go back to reference Reeve, B. B., Burke, L. B., Chiang, Y. P., Clauser, S. B., Colpe, L. J., Elias, J. W., et al. (2007). Enhancing measurement in health outcomes research supported by Agencies within the US Department of Health and Human Services. Quality of Life Research, 16(Suppl 1), 175–186. doi: 10.​1007/​s11136-007-9190-8. PubMedCrossRef Reeve, B. B., Burke, L. B., Chiang, Y. P., Clauser, S. B., Colpe, L. J., Elias, J. W., et al. (2007). Enhancing measurement in health outcomes research supported by Agencies within the US Department of Health and Human Services. Quality of Life Research, 16(Suppl 1), 175–186. doi: 10.​1007/​s11136-007-9190-8. PubMedCrossRef
43.
go back to reference Thissen, D., Chen, W.-H., & Bock, R. D. (2003). Multilog (version 7). Lincolnwood, IL: Scientific Software International. Thissen, D., Chen, W.-H., & Bock, R. D. (2003). Multilog (version 7). Lincolnwood, IL: Scientific Software International.
44.
go back to reference Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement No. 17. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement No. 17.
45.
go back to reference Bjorner, J. B., Smith, K. J., Stone, C., & Sun, X. (2007). IRTFIT: A macro for item fit and local dependence tests under IRT models. Lincoln, RI: QualityMetric. Bjorner, J. B., Smith, K. J., Stone, C., & Sun, X. (2007). IRTFIT: A macro for item fit and local dependence tests under IRT models. Lincoln, RI: QualityMetric.
46.
48.
go back to reference Han, K. T., & Hambleton, R. K. (2007). User’s manual: WinGen (Center for Educational Assessment Report No. 642). Amherst, MA: University of Massachusetts, School of Education. Han, K. T., & Hambleton, R. K. (2007). User’s manual: WinGen (Center for Educational Assessment Report No. 642). Amherst, MA: University of Massachusetts, School of Education.
49.
go back to reference Choi, S. W. (2008). Firestar: Computerized adaptive testing (CAT) simulation program for polytomous IRT models. Applied Psychological Measurement (in press). Choi, S. W. (2008). Firestar: Computerized adaptive testing (CAT) simulation program for polytomous IRT models. Applied Psychological Measurement (in press).
Metagegevens
Titel
Having a fit: impact of number of items and distribution of data on traditional criteria for assessing IRT’s unidimensionality assumption
Auteurs
Karon F. Cook
Michael A. Kallen
Dagmar Amtmann
Publicatiedatum
01-05-2009
Uitgeverij
Springer Netherlands
Gepubliceerd in
Quality of Life Research / Uitgave 4/2009
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-009-9464-4