Skip to main content
Top
Gepubliceerd in: Quality of Life Research 2/2014

01-03-2014

Is Rasch model analysis applicable in small sample size pilot studies for assessing item characteristics? An example using PROMIS pain behavior item bank data

Auteurs: Wen-Hung Chen, William Lenderking, Ying Jin, Kathleen W. Wyrwich, Heather Gelhorn, Dennis A. Revicki

Gepubliceerd in: Quality of Life Research | Uitgave 2/2014

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Purpose

Large samples are generally considered necessary for Rasch model to obtain robust item parameter estimates. Recently, small sample Rasch analysis was suggested as preliminary assessment of items’ psychometric properties. This study is to evaluate the Rasch analysis results using small sample sizes.

Methods

Ten PROMIS pain behavior items were used. Random samples of 30, 50, 100, and 250, and a targeted sample of 30 were drawn 10 times each from a total of 800 subjects. Rasch analysis was conducted for each of these samples and the full sample.

Results

In the full sample, there were 104 cases of extreme scores, no null categories, two incorrectly ordered items, and four misfit items. For samples of 250, 100, 50, 30, and targeted 30, the average numbers of extreme scores were 42.2, 17.1, 9.6, 6.1, and 1.2; the average numbers of null categories were 1.0, 3.2, 8.7, 13.4, and 8.3; the average numbers of items with incorrectly ordered item parameters were 0.1, 0.8, 2.9, 4.7, and 3.7; and the average numbers of items with fit residuals exceeding ±2.5 were 0.8, 0.3, 0.1, 0.2, and 0.3, respectively.

Conclusions

Rasch analysis based on small samples (≤50) identified a greater number of items with incorrectly ordered parameters than larger samples (≥100). However, fewer items were identified as misfitting. Results from small samples led to opposite conclusions from those based on larger samples. Rasch analysis based on small samples should be used for exploratory purposes with extreme caution.
Literatuur
1.
go back to reference Ring, L., Gross, C. R., & McColl, E. (2010). Putting the text back into context: Toward increased use of mixed methods for quality of life research. Quality of Life Research, 19(5), 613–615.PubMedCrossRef Ring, L., Gross, C. R., & McColl, E. (2010). Putting the text back into context: Toward increased use of mixed methods for quality of life research. Quality of Life Research, 19(5), 613–615.PubMedCrossRef
2.
go back to reference Klassen, A. C., Creswell, J., Plano Clark, V. L., Smith, K. C., & Meissner, H. I. (2012). Best practices in mixed methods for quality of life research. Quality of Life Research, 21(3), 377–380.PubMedCrossRef Klassen, A. C., Creswell, J., Plano Clark, V. L., Smith, K. C., & Meissner, H. I. (2012). Best practices in mixed methods for quality of life research. Quality of Life Research, 21(3), 377–380.PubMedCrossRef
3.
go back to reference Food and Drug Administration. (2009). Guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. Federal Register, 74(235), 65132–65133. Food and Drug Administration. (2009). Guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. Federal Register, 74(235), 65132–65133.
4.
go back to reference Hudgens, S., Globe, D., & Burgess, S. M. (2012). Utilization of Rasch measurement models for assessing validity: a mixed methods approach. Workshop presented at the International Society Pharmacoeconomics and Outcome Research (ISPOR) 17th Annual International Meeting, Washington, DC. Hudgens, S., Globe, D., & Burgess, S. M. (2012). Utilization of Rasch measurement models for assessing validity: a mixed methods approach. Workshop presented at the International Society Pharmacoeconomics and Outcome Research (ISPOR) 17th Annual International Meeting, Washington, DC.
5.
go back to reference Hudgens, S., Norquist, J., Wyrwich, K. W., Coons, S. J., & Lenderking, W. R. (Oct 24, 2012). Perspectives on mixed methods to assess content validity of a PRO measure. Presented at the Industry Advisory Committee Symposium, International Society for Quality of Life Research (ISOQOL) 19th annual conference, Budapest. Hudgens, S., Norquist, J., Wyrwich, K. W., Coons, S. J., & Lenderking, W. R. (Oct 24, 2012). Perspectives on mixed methods to assess content validity of a PRO measure. Presented at the Industry Advisory Committee Symposium, International Society for Quality of Life Research (ISOQOL) 19th annual conference, Budapest.
6.
go back to reference Linacre, J. M. (1994). Sample size and item calibrations stability. Rasch Measurement Transactions, 7(4), 328. Linacre, J. M. (1994). Sample size and item calibrations stability. Rasch Measurement Transactions, 7(4), 328.
7.
go back to reference Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85–106.PubMed Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85–106.PubMed
8.
go back to reference Reise, S. P., & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27(2), 133–144.CrossRef Reise, S. P., & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27(2), 133–144.CrossRef
9.
go back to reference Revicki, D. A., Chen, W. H., Harnam, N., Cook, K. F., Amtmann, D., Callahan, L. F., et al. (2009). Development and psychometric analysis of the PROMIS pain behavior item bank. Pain, 146(1–2), 158–169.PubMedCentralPubMedCrossRef Revicki, D. A., Chen, W. H., Harnam, N., Cook, K. F., Amtmann, D., Callahan, L. F., et al. (2009). Development and psychometric analysis of the PROMIS pain behavior item bank. Pain, 146(1–2), 158–169.PubMedCentralPubMedCrossRef
10.
go back to reference Muthén, L. K., & Muthén, B. (1998–2004). Mplus user’s guide (3rd ed.). Los Angeles, CA: Muthén & Muthén. Muthén, L. K., & Muthén, B. (1998–2004). Mplus user’s guide (3rd ed.). Los Angeles, CA: Muthén & Muthén.
11.
go back to reference Serlin, R. C., Mendoza, T. R., Nakamura, Y., Edwards, K. R., & Cleeland, C. S. (1995). When is cancer pain mild, moderate or severe? Grading pain severity by its interference with function. Pain, 61(2), 277–284.PubMedCrossRef Serlin, R. C., Mendoza, T. R., Nakamura, Y., Edwards, K. R., & Cleeland, C. S. (1995). When is cancer pain mild, moderate or severe? Grading pain severity by its interference with function. Pain, 61(2), 277–284.PubMedCrossRef
12.
go back to reference Andrich, D., Sheridan, B., & Lou, G. (2009). RUMM2030. Perth, Australia: RUMM Laboratory. Andrich, D., Sheridan, B., & Lou, G. (2009). RUMM2030. Perth, Australia: RUMM Laboratory.
13.
go back to reference Andrich, D. (2004). Controversy and the Rasch model: a characteristic of incompatible paradigms? Medical Care, 42(1 Suppl), I7–16.PubMed Andrich, D. (2004). Controversy and the Rasch model: a characteristic of incompatible paradigms? Medical Care, 42(1 Suppl), I7–16.PubMed
14.
go back to reference Wright, B. D. (1999). Fundamental measurement for psychology. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement: What every educator and psychologist should know (pp. 65–104). New Jersey: Lawrence Erlbaum Associates. Wright, B. D. (1999). Fundamental measurement for psychology. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement: What every educator and psychologist should know (pp. 65–104). New Jersey: Lawrence Erlbaum Associates.
15.
go back to reference Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606.CrossRef Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606.CrossRef
16.
go back to reference Greenwood, P. E., & Nihulin, M. S. (1996). A guide to Chi square testing. New York, NY: Wiley. Greenwood, P. E., & Nihulin, M. S. (1996). A guide to Chi square testing. New York, NY: Wiley.
Metagegevens
Titel
Is Rasch model analysis applicable in small sample size pilot studies for assessing item characteristics? An example using PROMIS pain behavior item bank data
Auteurs
Wen-Hung Chen
William Lenderking
Ying Jin
Kathleen W. Wyrwich
Heather Gelhorn
Dennis A. Revicki
Publicatiedatum
01-03-2014
Uitgeverij
Springer International Publishing
Gepubliceerd in
Quality of Life Research / Uitgave 2/2014
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-013-0487-5

Andere artikelen Uitgave 2/2014

Quality of Life Research 2/2014 Naar de uitgave