Response shift influenced estimates of change in health-related quality of life poststroke

Abstract

Objective

The interpretability of changes in perceived health status over time is threatened if people experience a response shift. This study assessed whether the recovery process following stroke altered individuals' perceptions of past health status and the impact that change in internal standards (response shift) had on ratings over time. We hypothesized that individuals with stroke would experience changes in internal standards, not experienced by the control group. Two other hypotheses related to objective criterion measures also were tested.

Study design and setting

Individuals were recruited through a randomized trial of acute poststroke care. Health status was evaluated at baseline (within the first week poststroke), 6 and 24 weeks later using the EQ VAS. At 6 and 24 weeks, subjects were asked to retrospectively re-evaluate their health status for the preceding evaluation using the then test technique.

Results

The pattern of mean scores was indicative of changes in internal standards among individuals with stroke but not for the control group. Memory had an impact on estimates of response shift. Hypotheses related to the objective criterion measures were not supported.

Conclusion

The results suggest that there was a change in internal standards, and that measures of improvement in health status are different based on prospective as compared to retrospective ratings. Further understanding of the impact of recall on the assessment of response shift using the then test is needed to validate the use of this technique.

Introduction

Assessments of health-related quality of life (HRQL) have played an important role in understanding the impact of stroke on an individual's health and well being [1], [2], [3], [4], [5], [6]. This information is valuable because the patient's opinion of their health is often different from that of medical professionals and family members. There may also be changes in health that cannot be reflected by objective clinical measures alone. It is this advantageous aspect of HRQL measures, their self-report nature, which presents a challenge for using them as an outcome for assessing change over time or between groups. Because measures of HRQL are completed from the perspective of the individual, measurement of this construct is modified by psychologic phenomena such as coping and adaptation. Changes in HRQL may not only be due to changes in the disease or response to treatment, but a part of that change may be attributed to what has been referred to as response shift. The working definition of response shift is “a change in one's evaluation of a target construct as a result of: (a) a change in the respondent's internal standards of measurement (i.e., scale recalibration), (b) a change in the respondent's values (i.e., the importance of component domains constituting the target construct), or (c) a redefinition of the target construct (i.e., reconceptualization)” [7].

When assessments of HRQL are made at various points in time poststroke or before and after an intervention, the assumption is usually that the individual's point of reference and calibration of the measurement scale that is being used to assess HRQL remains the same. Given the nature of stroke, this is most likely not the case. Stroke comes on suddenly, and is followed by a recovery process that is most rapid during the first 5 weeks, but may continue until 6 months poststroke. Therefore, most individuals with stroke are undergoing changes in physical health.

These changes that occur over time poststroke (or as the result of an intervention) might influence an individuals' internalized standard of judgment of what value on a scale reflects excellent or poor levels of HRQL. For example, a person who shortly after the stroke was unable to walk 50 m without assistance, who is now able to walk independently in the community, might think back and judge their initial level of functioning quite differently than they did immediately poststroke. Although response shift is a natural way by which individuals who are confronted with a serious illness adapt and cope with changes in physical health [8], [9], it influences objective measures of change of HRQL. It becomes difficult to distinguish objective change due to the illness or an intervention from response shift. Conventional comparisons of mean scores before and after an elapsed period of time do not differentiate between these types of change.

Several investigators have turned to response shift as an explanation of paradoxes found in HRQL research. These include instances when populations with chronic diseases report levels of HRQL that are equivalent to healthy controls [2], [10], [11], when patients rate their HRQL better than their providers [12], [13], and when discrepancies between objective ratings and self ratings of health occur [14], [15], [16].

This study focuses on the assessment of changes in internal standards poststroke (i.e., scale recalibration). The most widely used method for assessing this form of response shift is the then test, originally suggested by Howard and colleagues [17]. This method not only uses an individuals' response to a preintervention (first evaluation) and postintervention (second evaluation) assessment, but also uses a then measure, which is a retrospective rating of the first evaluation. The then measure is completed at the same time as the postintervention evaluation, and asks subjects to complete the self-report measure in reference to how they perceive themselves to have been at the preintervention. As the post-test and the then measure are administered at the same time, they are presumed to be completed with the same internal standard. The mean difference between the pretest and then test scores provides an estimate of the direction and the magnitude of the response-shift effect. Although this method is intended to assess changes in internal standards, implicit in this change may be the notion of reconceptualization of HRQL. If new experiences result in a recalibration of the measurement scale, it is likely that the anchors and intervals will have different meanings [18], [19].

The then test approach has been applied most extensively for assessing changes in internal standards elicited by educational interventions. Results from these studies have provided supporting evidence for the greater validity of self-reported measures of change that use retrospective ratings of pretests to remove response-shift effects compared to the use of traditional pretests [17], [19], [20], [21], [22], [23]. More recent applications of this method in the health field have also indicated that response shift effects influence conventional pretest–post-test comparisons [8], [24], [25]. Sprangers et al. [24] applied the then test in a group of 99 cancer patients undergoing radiotherapy and found that there was a response shift effect in the area of fatigue for patients experiencing diminishing levels of fatigue and patients facing early stages of adaptation to increased levels of fatigue [24]. Other studies that have assessed changes in internal standards among patients with cancer [25], [26] and end-stage renal disease [8] have found similar results.

There have been no formal assessments of changes in internal standards in the stroke population. The objective of this study was to determine whether the recovery experience following stroke altered individuals' perception of past health status. We hypothesized that, given the expected improvements in physical health, individuals with stroke would, on average, retrospectively report worse baseline health status (report lower then test ratings), when rated retrospectively, at 6 weeks and 24 weeks poststroke than what they originally reported it to be at the previous evaluation. After having survived an event like stroke (20% of individuals die), a health event that people fear the most, the majority of individuals tend to over-rate their health, especially those who are discharged home. We hypothesized that as time goes on and people's functional ability improves, that individuals would retrospectively reevaluate how they were in the past. Initially they would be grateful for survival and some function, but later in the recovery process they realize that they were quite bad in the past. We further hypothesized that as the members of the control group were not expected to experience changes in physical health, they would retrospectively rate their health the same as they did at the first evaluation.

Two additional hypotheses were tested with respect to criterion objective measures of physical ability. First, we hypothesized that individuals who improved on the objective measures would retrospectively lower their previous health status (i.e., lower then test ratings) to a greater extent compared to those who deteriorated or did not change. Second, it was hypothesized that those who improved on the objective measures would also have greater improvements in response shift adjusted change compared to those who deteriorated or did not change, and that this pattern would not be as strong for conventional assessments of change.

For exploratory purposes, a secondary objective was to assess which stroke and sociodemographic characteristics were associated with the direction and magnitude of response shift during the first 6 months poststroke.