Original ArticleThe structural equation modeling technique did not show a response shift, contrary to the results of the then test and the individualized approaches
Introduction
Health-related quality-of-life (HRQL) has emerged as an important construct for measuring the outcome of disease processes and interventions in the health care system because it captures, from the individuals point of view, aspects of physical, emotional, social, and psychologic well-being parsimoniously [1], [2]. One of the challenges in using HRQL as an outcome is that as people experience changes in their physical health owing to improvement or to deterioration, or owing to an intervention, they change their values and rerate the importance of basic elements of HRQL. In the scientific literature, this natural and advantageous process has been termed response shift. The working definition of response shift is “a change in one's evaluation of a target construct as a result of: (a) a change in the respondent's internal standards of measurement (i.e., scale recalibration); (b) a change in the respondents values (i.e., the importance of component domains constituting the target construct); or (c) a redefinition of the target construct (i.e., reconceptualization)” [3].
Stroke is one condition where response shift is likely. After a stroke, individuals experience a sudden loss of function, which recovers at a variable rate over the ensuing days and weeks. Recovery usually plateaus by 3 months, and individuals realize that they are likely to remain at this level of functioning. The health care system offers most services early on after stroke, and emphasizes improving impairments and activity limitations. Beyond 3 months, formal rehabilitation ceases and patients are encouraged in community participation with the ultimate aim of reaching an optimal HRQL. Therefore, tracking HRQL over time is a challenge because of the potential for response shift.
As HRQL is increasingly becoming part of the evaluation profile for interventions poststroke, particularly those interventions involving health services delivery, developing an estimator of HRQL that differentiates between objective change and changes in standards, conceptualization, and values is essential for the interpretation of the results. The strength of randomized trials is that balance is achieved at the outset on measured and unmeasured variables, which would include conceptualization of HRQL and internal standards. However, in trials where the intervention involves components of care and support by a team of health professionals (e.g., trials of rehabilitation services, continuity of care, home care, family support, community interventions, etc.), subjects in the treatment arm may be provided with new information about stroke and its recovery and ways of coping with the consequences of the disease. As a result, the intervention arm may have a response shift induced leading to a differential response shift in the two groups.
Response shift may attenuate or exaggerate findings from clinical trials that use HRQL as an end point. Comparability between two measurements taken at two points in time, or between two different groups, requires a common metric between the two sets of scores [4]. If there is a differential response shift then the assumption that both groups, on average, are equal except for the effect of the intervention is no longer true. In this situation, it becomes difficult to disentangle which component of change is due to response shift, vs. actual change in the construct (here HRQL).
In a recent study by Sulch [5], the implementation of an integrated care pathway throughout the first 6 months poststroke was expected to improve HRQL, as evaluated by the EQ-5D, more than the usual care provided by a multidisciplinary team. The results showed the opposite with those in the usual care group reporting higher HRQL. This unexpected finding may have been a consequence of the intervention group recalibrating their value on the measure over time with the accrual of new information, and extra guidance from the medical staff.
Therefore, studies that will evaluate change in HRQL where an intervention or disease process is likely to initiate changes in standards, conceptualization, or values, need to incorporate methods for evaluating response shift. There are currently no guidelines available for researchers on how to measure response shift in HRQL evaluations and to account for response shift in the estimation of change. The purpose of this study is to outline recommendations to understand how change in HRQL can be estimated in the presence of response shift. To develop these recommendations three methodologic approaches for evaluating response shift were compared among individuals with stroke.
Section snippets
Methods
A brief description of each method used to evaluate response shift will be given below, as details of the three approaches used to evaluate response shift have been presented elsewhere [6], [7]. The methods were compared in terms of feasibility and results regarding response shift. The data collection for the three approaches was incorporated into a randomized controlled trial that assessed the impact of a home intervention on stroke outcome. Eligible consenting subjects with stroke had
Analyses
To evaluate feasibility we compared the proportion of subjects who were able to complete the three approaches. We also compared whether the approaches provide consistent answers regarding the occurrence of response shift poststroke. In addition, because the then test and the individualized method are performed at the individual level and not at the group level, as is the case with the structural equation approach, we were able to make a direct comparison between change in areas on the PGI and
Results
Table 1 presents a summary of the proportion of subjects who participated in the three methodologic approaches, and an overview of the results regarding the occurrence of response shift. Seventy-nine percent of subjects had complete information on the SF-36, which reflected the proportion of subjects who were included in the structural equation modeling analysis to evaluate response shift. For both evaluation periods (baseline to 6 weeks, 6 weeks to 24 weeks) over 95% of subjects had complete
Discussion
Methods for evaluating response shift will be an important part of HRQL evaluations when persons are likely to experience a reconceptualization and/or change in internal standards. A key element for selecting an approach within the context of a parallel study design (clinical trial or cohort study) is the time required and the proportion of subjects in the study sample who will be able to complete the procedure. The structural equation modeling technique is clearly the easiest to apply because
Summary and conclusions
Despite the complexity of evaluating HRQL to capture the perspective of the individual, it is an essential component of medical research. Testing of methodologic approaches to evaluate response shift in HRQL research is relatively new. Comparisons of approaches at this early stage are important to support the validity of the various techniques and to identify feasible approaches to incorporate in ongoing studies.
The work presented in this article has highlighted and contrasted the application
Acknowledgments
This project was supported by the Canadian Institute of Health Research (CIHR) and the Fonds de la recherche en santé du Québec (FRSQ).
References (33)
- et al.
Integrating response shift into health-related quality of life research: a theoretical model
Soc Sci Med
(1999) - et al.
The impact of response shift influenced estimates of change in health-related quality of life post-stroke
J Clin Epidemiol
(2004) - et al.
Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research
Soc Sci Med
(1999) Understanding and using health-related quality of life instruments within clinical research studies
Patient-reported outcome (PROs): an organizing tool for concepts, measures, and applications. Quality of life news letter
(2003)- et al.
How should we measure “change”—Or should we?
Psychol Bull
(1970) - et al.
Integrated care pathways and quality of life on a stroke rehabilitation unit
Stroke
(2002) - Ahmed S, Mayo NE, Corbiere M, Wood-Dauphinee S, Hanley J, Cohen R. Change in quality of life in people with stroke over...
The use of analysis of covariance structures to assess beta and gamma change
Multivariate Behav Res
(1982)- et al.
Alpha, beta, and gamma change in evaluation research: a structural equation approach
J Appl Psychol
(1988)