Original Article
The structural equation modeling technique did not show a response shift, contrary to the results of the then test and the individualized approaches

https://doi.org/10.1016/j.jclinepi.2005.03.003Get rights and content

Abstract

Background and Objective

Persons experiencing changes in their physical health may change their values and rerate the importance of basic elements of health-related quality of life (HRQL), a process known as response shift. Developing an estimator of HRQL that differentiates between objective change and response shift is essential for the interpretation of the results. The purpose of the present article was to contrast three methodologic approaches for evaluating response shift to develop a proposed set of HRQL measurement recommendations under circumstances where response shift is expected to occur.

Methods

The three approaches compared were a structural equation modeling (SEM) technique, the then test, and an individualized approach. The data collection procedures for these methods were incorporated into a poststroke randomized controlled trial.

Results

The SEM did not show a response shift, contrary to the results of the then test and the individualized approaches. We discuss factors that effect the selection of a methodologic approach including feasibility, subjects' memory and more advanced cognitive tasks, and whether response shift was evaluated at the group or individual level.

Conclusion

The evaluation of response shift is an integral part of HRQL evaluations, and further comparisons between methodologic approaches are needed.

Introduction

Health-related quality-of-life (HRQL) has emerged as an important construct for measuring the outcome of disease processes and interventions in the health care system because it captures, from the individuals point of view, aspects of physical, emotional, social, and psychologic well-being parsimoniously [1], [2]. One of the challenges in using HRQL as an outcome is that as people experience changes in their physical health owing to improvement or to deterioration, or owing to an intervention, they change their values and rerate the importance of basic elements of HRQL. In the scientific literature, this natural and advantageous process has been termed response shift. The working definition of response shift is “a change in one's evaluation of a target construct as a result of: (a) a change in the respondent's internal standards of measurement (i.e., scale recalibration); (b) a change in the respondents values (i.e., the importance of component domains constituting the target construct); or (c) a redefinition of the target construct (i.e., reconceptualization)” [3].

Stroke is one condition where response shift is likely. After a stroke, individuals experience a sudden loss of function, which recovers at a variable rate over the ensuing days and weeks. Recovery usually plateaus by 3 months, and individuals realize that they are likely to remain at this level of functioning. The health care system offers most services early on after stroke, and emphasizes improving impairments and activity limitations. Beyond 3 months, formal rehabilitation ceases and patients are encouraged in community participation with the ultimate aim of reaching an optimal HRQL. Therefore, tracking HRQL over time is a challenge because of the potential for response shift.

As HRQL is increasingly becoming part of the evaluation profile for interventions poststroke, particularly those interventions involving health services delivery, developing an estimator of HRQL that differentiates between objective change and changes in standards, conceptualization, and values is essential for the interpretation of the results. The strength of randomized trials is that balance is achieved at the outset on measured and unmeasured variables, which would include conceptualization of HRQL and internal standards. However, in trials where the intervention involves components of care and support by a team of health professionals (e.g., trials of rehabilitation services, continuity of care, home care, family support, community interventions, etc.), subjects in the treatment arm may be provided with new information about stroke and its recovery and ways of coping with the consequences of the disease. As a result, the intervention arm may have a response shift induced leading to a differential response shift in the two groups.

Response shift may attenuate or exaggerate findings from clinical trials that use HRQL as an end point. Comparability between two measurements taken at two points in time, or between two different groups, requires a common metric between the two sets of scores [4]. If there is a differential response shift then the assumption that both groups, on average, are equal except for the effect of the intervention is no longer true. In this situation, it becomes difficult to disentangle which component of change is due to response shift, vs. actual change in the construct (here HRQL).

In a recent study by Sulch [5], the implementation of an integrated care pathway throughout the first 6 months poststroke was expected to improve HRQL, as evaluated by the EQ-5D, more than the usual care provided by a multidisciplinary team. The results showed the opposite with those in the usual care group reporting higher HRQL. This unexpected finding may have been a consequence of the intervention group recalibrating their value on the measure over time with the accrual of new information, and extra guidance from the medical staff.

Therefore, studies that will evaluate change in HRQL where an intervention or disease process is likely to initiate changes in standards, conceptualization, or values, need to incorporate methods for evaluating response shift. There are currently no guidelines available for researchers on how to measure response shift in HRQL evaluations and to account for response shift in the estimation of change. The purpose of this study is to outline recommendations to understand how change in HRQL can be estimated in the presence of response shift. To develop these recommendations three methodologic approaches for evaluating response shift were compared among individuals with stroke.

Section snippets

Methods

A brief description of each method used to evaluate response shift will be given below, as details of the three approaches used to evaluate response shift have been presented elsewhere [6], [7]. The methods were compared in terms of feasibility and results regarding response shift. The data collection for the three approaches was incorporated into a randomized controlled trial that assessed the impact of a home intervention on stroke outcome. Eligible consenting subjects with stroke had

Analyses

To evaluate feasibility we compared the proportion of subjects who were able to complete the three approaches. We also compared whether the approaches provide consistent answers regarding the occurrence of response shift poststroke. In addition, because the then test and the individualized method are performed at the individual level and not at the group level, as is the case with the structural equation approach, we were able to make a direct comparison between change in areas on the PGI and

Results

Table 1 presents a summary of the proportion of subjects who participated in the three methodologic approaches, and an overview of the results regarding the occurrence of response shift. Seventy-nine percent of subjects had complete information on the SF-36, which reflected the proportion of subjects who were included in the structural equation modeling analysis to evaluate response shift. For both evaluation periods (baseline to 6 weeks, 6 weeks to 24 weeks) over 95% of subjects had complete

Discussion

Methods for evaluating response shift will be an important part of HRQL evaluations when persons are likely to experience a reconceptualization and/or change in internal standards. A key element for selecting an approach within the context of a parallel study design (clinical trial or cohort study) is the time required and the proportion of subjects in the study sample who will be able to complete the procedure. The structural equation modeling technique is clearly the easiest to apply because

Summary and conclusions

Despite the complexity of evaluating HRQL to capture the perspective of the individual, it is an essential component of medical research. Testing of methodologic approaches to evaluate response shift in HRQL research is relatively new. Comparisons of approaches at this early stage are important to support the validity of the various techniques and to identify feasible approaches to incorporate in ongoing studies.

The work presented in this article has highlighted and contrasted the application

Acknowledgments

This project was supported by the Canadian Institute of Health Research (CIHR) and the Fonds de la recherche en santé du Québec (FRSQ).

References (33)

  • J. Schaubroeck

    Confirmatory factor analytic procedures for assessing change during organizational entry

    J Appl Psychol

    (1989)
  • R.B. Kline

    Principles and practice of structural equation modeling

    (1998)
  • M.K. Lindell

    Equivocality of factor incongruence as an indicator of type of change in OD interventions

    Acad Manage Rev

    (1980)
  • J.E. Ware et al.

    Health survey: manual & interpretation guide

    (1993)
  • P. Dorman et al.

    Qualitative comparison of the reliability of health status assessments with the EuroQol and SF-36 questionnaires after stroke. United Kingdom Collaborators in the International Stroke Trial

    Stroke

    (1998)
  • P.J. Dorman et al.

    Is the EuroQol a valid measure of health-related quality of life after stroke?

    Stroke

    (1997)
  • Cited by (0)

    View full text