Skip to main content
main-content
Top

Tip

Swipe om te navigeren naar een ander artikel

Gepubliceerd in: Quality of Life Research 8/2007

01-10-2007 | Original Paper

Controlling for occasion-specific effects when assessing the test–retest reliability of self-report health questionnaires

Auteurs: Joseph A. Olsen, Daniel A. Bloch, George J. Bloch

Gepubliceerd in: Quality of Life Research | Uitgave 8/2007

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Objective

This study proposes a method for self-report health questionnaires to adjust test–retest reliability for changes during the test–retest interval based on an external measure, and to distinguish such changes from random response errors.

Methods

In our application, eighty participants completed the Symptoms of Illness Checklist (SIC) on two occasions, two weeks apart, immediately before interviews given on each occasion by one of two physicians in a crossover design. The physician interview scores served as external measures, and structural equation modeling was used to estimate the parameters of a model that corrected for the occasion-specific effect of participants’ responses using information from the interviews.

Results

Correcting for changes in symptoms during the test–retest interval increased SIC test–retest reliability from .744 to .804 and significantly improved model fit (χ2 diff (1) = 30.78, p < .001).

Conclusions

The results suggest methods that can improve the evaluation of self-report health questionnaire test–retest reliability by identifying changes using an external measure, and distinguishing these from random response errors; these increased the estimated SIC test–retest reliability and indicated that the SIC was indeed able to measure changes over the studied time interval. This method can be applied across a broad range of questionnaires.
Bijlagen
Alleen toegankelijk voor geautoriseerde gebruikers
Voetnoten
1
AMOS allows the model to be specified graphically in the form of a path diagram—in the present case the illustration presented in Fig. 1, but without the triangle and the lines emanating from it (these intercepts are estimated by default in AMOS). If the reader wishes to obtain a copy of the AMOS program file used in the present study, please contact Joseph Olsen at joseph_olsen@byu.edu.
 
Literatuur
1.
go back to reference Schmidt, F. L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs. Psychological Methods, 8, 206–224. CrossRef Schmidt, F. L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs. Psychological Methods, 8, 206–224. CrossRef
2.
go back to reference Laenen, A., Vangeneugden, T., Geys, H., & Molenberghs, G. (2006). Generalized reliability estimation using repeated measurements. British Journal of Mathematical and Statistical Psychology, 59, 113–131. CrossRef Laenen, A., Vangeneugden, T., Geys, H., & Molenberghs, G. (2006). Generalized reliability estimation using repeated measurements. British Journal of Mathematical and Statistical Psychology, 59, 113–131. CrossRef
3.
go back to reference Schuck, P. (2004). Assessing reproducibility for interval data in health-related quality of life questionnaires: Which coefficient should be used? Quality of Life Research, 13, 571–586. CrossRef Schuck, P. (2004). Assessing reproducibility for interval data in health-related quality of life questionnaires: Which coefficient should be used? Quality of Life Research, 13, 571–586. CrossRef
4.
go back to reference Becker, G. (2000). How important is transient error in estimating reliability? Going beyond simulation studies. Psychological Methods, 5, 370–379. CrossRef Becker, G. (2000). How important is transient error in estimating reliability? Going beyond simulation studies. Psychological Methods, 5, 370–379. CrossRef
5.
go back to reference Green, S. B. (2003). A coefficient Alpha for test–retest data. Psychological Methods, 8, 88–101. CrossRef Green, S. B. (2003). A coefficient Alpha for test–retest data. Psychological Methods, 8, 88–101. CrossRef
6.
go back to reference Vautier, S., & Jmel, S. (2003). Transient error or specificity? An alternative to the staggered equivalent split-half procedure. Psychological Methods, 8, 225–238. CrossRef Vautier, S., & Jmel, S. (2003). Transient error or specificity? An alternative to the staggered equivalent split-half procedure. Psychological Methods, 8, 225–238. CrossRef
7.
go back to reference Raykov, T., & Penev S. (2005). Estimating the reliability for multiple component measuring instruments in test–retest designs. British Journal of Mathematical and Statistical Psychology, 58, 285–299. CrossRef Raykov, T., & Penev S. (2005). Estimating the reliability for multiple component measuring instruments in test–retest designs. British Journal of Mathematical and Statistical Psychology, 58, 285–299. CrossRef
8.
go back to reference Sturman, M. C., Cheramie, R. A., & Cashen, L. H. (2005). The impact of job complexity and performance measurement on the temporal consistency, stability, and test–retest reliability of employee job performance ratings. Journal of Applied Psychology, 90, 269–283. CrossRef Sturman, M. C., Cheramie, R. A., & Cashen, L. H. (2005). The impact of job complexity and performance measurement on the temporal consistency, stability, and test–retest reliability of employee job performance ratings. Journal of Applied Psychology, 90, 269–283. CrossRef
9.
go back to reference Watson, D. (2004). Stability versus change, dependability versus error: Issues in the assessment of personality over time. Journal of Research in Personality, 38, 319–350. CrossRef Watson, D. (2004). Stability versus change, dependability versus error: Issues in the assessment of personality over time. Journal of Research in Personality, 38, 319–350. CrossRef
10.
go back to reference Stowell, J. R., & Bloch, G. J. (2002, April). The symptoms of illness checklist (SIC): A relation between health and stress. (Paper presented at the Rocky Mountain Psychological Association, Abstract 180.). Stowell, J. R., & Bloch, G. J. (2002, April). The symptoms of illness checklist (SIC): A relation between health and stress. (Paper presented at the Rocky Mountain Psychological Association, Abstract 180.).
11.
go back to reference Stowell, J. R., Hedges, D. W., Ghambaryan. A., Key, C., & Bloch, G. J. (Submitted, 2007). Validation of the Symptoms of Illness Checklist (SIC) as a tool for health psychology research. Journal of Health Psychology. Stowell, J. R., Hedges, D. W., Ghambaryan. A., Key, C., & Bloch, G. J. (Submitted, 2007). Validation of the Symptoms of Illness Checklist (SIC) as a tool for health psychology research. Journal of Health Psychology.
12.
go back to reference Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.
13.
go back to reference McDowell, I., & Newell, C. (1996). Measuring health: A guide to rating scales and questionnaires. New York: Oxford Press. McDowell, I., & Newell, C. (1996). Measuring health: A guide to rating scales and questionnaires. New York: Oxford Press.
14.
go back to reference Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., et al. (1993). The European organization for research and treatment of cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85, 365–376. CrossRef Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., et al. (1993). The European organization for research and treatment of cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85, 365–376. CrossRef
15.
go back to reference Ware, J. E., & Sherbourne, C. D. (1992). The MOS 36-item Short-Form Health Survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30, 473–483. CrossRef Ware, J. E., & Sherbourne, C. D. (1992). The MOS 36-item Short-Form Health Survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30, 473–483. CrossRef
Metagegevens
Titel
Controlling for occasion-specific effects when assessing the test–retest reliability of self-report health questionnaires
Auteurs
Joseph A. Olsen
Daniel A. Bloch
George J. Bloch
Publicatiedatum
01-10-2007
Uitgeverij
Springer Netherlands
Gepubliceerd in
Quality of Life Research / Uitgave 8/2007
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-007-9246-9