This study proposes a method for self-report health questionnaires to adjust test–retest reliability for changes during the test–retest interval based on an external measure, and to distinguish such changes from random response errors.
In our application, eighty participants completed the Symptoms of Illness Checklist (SIC) on two occasions, two weeks apart, immediately before interviews given on each occasion by one of two physicians in a crossover design. The physician interview scores served as external measures, and structural equation modeling was used to estimate the parameters of a model that corrected for the occasion-specific effect of participants’ responses using information from the interviews.
Correcting for changes in symptoms during the test–retest interval increased SIC test–retest reliability from .744 to .804 and significantly improved model fit (χ2 diff (1) = 30.78, p < .001).
The results suggest methods that can improve the evaluation of self-report health questionnaire test–retest reliability by identifying changes using an external measure, and distinguishing these from random response errors; these increased the estimated SIC test–retest reliability and indicated that the SIC was indeed able to measure changes over the studied time interval. This method can be applied across a broad range of questionnaires.