Methodological Innovation
Case management poststroke did not induce response shift: the value of residuals

https://doi.org/10.1016/j.jclinepi.2009.03.020Get rights and content

Abstract

Objectives

To estimate the extent to which different methods of assessing response shift lead to different conclusions about its presence.

Study Design and Setting

A reanalysis of a randomized clinical trial involving 190 persons poststroke to evaluate the effectiveness of a case management intervention aimed at assisting persons with stroke to make the transition from acute care to home.

Results

Response shift was found to be nondifferential across groups and was therefore ruled out as an explanation for why the case management intervention showed no impact on patients' perceptions of their health. The results of a then-test did not show differential recalibration overtime. Factor analysis did not support reconceptualization response shift in either group but did find weak evidence for reprioritization response shift. An analysis of residuals indicated that approximately 50% of study subjects in each group experienced response shift that is probably close to reconceptualization.

Conclusion

A framework for assessing response shift was proposed and investigators planning trials of interventions targeting patient-reported outcomes should build into the trial methods for response shift investigation. In trials of interventions likely to induce response shift, before concluding about intervention-induced change, response shift should be ruled out by using a combination of design and statistical approaches.

Introduction

What is new?

When using patient reported outcomes (PROs) to evaluate the effects of an intervention, an outcome may change owing to persons reconceptualizing or recalibrating the outcome rather than truly changing. This article provides a statistical framework for examining this phenomenon in the context of a randomized clinical trial.

Health care interventions targeting improvement of patient-reported outcomes, such as symptoms, function, and health-related quality of life (HRQL) can induce both true change and response shift. Response shift is defined as changes in a person's self-evaluation, resulting from changes in internal standards or recalibration of the measurement scale; changes in the definition or conceptualization of the construct; and/or changes in values or in the prioritization of domains within the construct [1]. Stroke is one condition where response shift is a likely scenario because of the sudden onset of loss of function that recovers at a variable rate over the ensuing days to months. The health care system offers most services early on after stroke and emphasizes improving impairments and activity limitations. Beyond 3 months, formal rehabilitation ceases and persons are encouraged in community participation with the ultimate aim of reaching an optimal HRQL. This shift in focus from impairment to participation and quality of life would induce response shift as stroke recoverers learn to value what they can do despite impairments rather than focusing on the limitation itself. Evaluating interventions is complicated by spontaneous recovery, change in outcome emphasis, and response shift. For example, in trials where the intervention involves components of care and support by a team of health professionals (e.g., trials of rehabilitation services, continuity of care, home care, family support, community interventions, and so forth), the intervention may act as a catalyst for participants to recalibrate, reconceptualize, or reprioritize constructs that are included in measures such as of HRQL.

In a trial, both groups at randomization likely start with the same conceptualization of the outcome and the same internal standard of measurement. Through the intervention, the treatment arm may acquire new information and knowledge about stroke (prevention, consequences) and ways of coping with the consequences of the disease (functional or social limitations), and may be encouraged to interact with others who are going through the same experience (support groups that may initiate social comparisons, and so on). In this situation, changes because of response shift and the designed-in components of the intervention become difficult to disentangle. In many instances, response shift is integral to the intervention and there is really no need to untangle the separate effects. However, if elements of the intervention act to improve some outcomes but response shift acts in the opposite direction on other outcomes, then separating out these effects becomes crucial.

The literature on early supported discharge poststroke offers several examples where response shift could have occurred [2], [3], [4], [5]. For example, three of these studies found that the intervention group as compared with the control group had significantly higher scores on objective clinical measures, performance-based measures, or measures of activities of daily living, but not on HRQL [2], [3], [5]. The intervention offered by the home care team may have acted as a catalyst for the initiation of response shift in the treatment group, perhaps making them more aware earlier of the impact of stroke on their lives as therapy and care were offered at home. This effect may have been delayed for the comparison group and this differential effect would make the two groups incomparable. A simple comparison of mean scores, as is traditionally done, would attenuate or exaggerate differences in HRQL between the groups, or the groups would appear to have similar levels of HRQL, as was the case for these studies on early supported discharge. If assessments of change in HRQL do not account for possible response shift, results could be misleading regarding the added benefit of an intervention and in turn could affect health care policy planning. Although these and other trials [6] suggest that response shift can occur, methods for directly assessing response shift were not included.

Methods for response shift investigation are often classified as design-based or statistical but an alternate classification might be individual- or group-based [7]. Design-based methods include the then-test [8], [9] and personalized interviewing [10] and yield information on individuals but they must be put in place in the study from the outset.

Statistical methods up until recently were mainly group-based and assessed average response shift using techniques such as structural equation modeling (SEM) [11], a technique requiring large sample sizes. Mayo et al. [12] recently introduced a statistical approach assessing response shift at an individual level by examining the difference between a patient-reported outcome and what would have been predicted for this person based on other measured variables. The pattern of these residuals overtime was used to infer response shift.

We recently evaluated, through a randomized trial, an intervention aimed at assisting persons with stroke to make the transition from acute care to home. In this study, a nurse, through home visits and telephone monitoring, provided nursing interventions that included active surveillance of health status, information about stroke, medication management, assistance with accessing needed health care services, active listening, and family support [13]. A control group received no additional help with making the transition from hospital to home.

There were no statistically significant differences between groups on the primary outcome measure, self-rating of HRQL, nor on any of the secondary outcome measures or health services utilization. For this population, there was no evidence that this type of passive case management inferred any added benefit in terms of improvement in perceived health or reduction in health services utilization and stroke impact, than usual postdischarge management.

The infrastructure put in place for this trial permitted, for a subset of the sample, an evaluation of several aspects of response shift [7], [9], [14]. However, the effect of response shift on the trial results has not yet been examined. Given that methods for detecting response shift are few and those that are available are not part of the usual statistical approaches to data analysis, the specific objective of this article is to estimate, in one data set, the extent to which different methods of assessing response shift lead to different conclusions about the presence of response shift. We hypothesized that the Case Management group would experience a greater degree of recalibration as measured by the then-test than the Usual Care group. We also hypothesized that case management would induce response shift that would initially be negative (at 6 weeks) because the nurse manager would make people more aware of their need for care but by 6 months the intervention group would rebound, with people reporting higher levels of health than predicted by their stroke-related disability. We hypothesized that both reconceptualization and reprioritization response shifts would occur differentially between treatment groups and this would partially explain why there was no difference between the two groups. The ultimate aim of this project was to propose a framework for investigating response shift, in the context of a clinical trial evaluating an intervention with the potential to invoke a differential response shift between treatment groups.

Section snippets

Methods

This study is a secondary analysis of data arising from a randomized clinical trial [13] designed to test whether people who have had an acute stroke benefit from having a case manager after hospital discharge. The outcomes evaluated were HRQL and health service use. The comparison group was people receiving usual posthospital care. Data were available at randomization (on average 12 days poststroke), after 6 weeks of intervention, and then at 6 months poststroke.

Results

Table 1 shows the characteristics of the total sample from the original trial [13] and the subsample with the data required for the different analyses investigating response shift. Also shown are the values on the main outcome the EQ-VAS and its predictive variables at three points in time. EQ predicted is also given. Data on the SF-36 were not collected at randomization as this was done on average 12 days poststroke while participants were still in hospital, making it difficult to respond

Discussion

When using a patient-reported outcome in a trial where the intervention could theoretically cause people to reappraise specific aspects of their health that are captured within the outcome measure, it is wise to suspect response shift. In this study, response shift was not an unlikely hypothesis because the persons assigned to the stroke case manager received home visits and telephone contacts for a period of 6 weeks. The interactions and interventions that the nurse case manager carried out

References (36)

  • C.E. Schwartz

    Teaching coping skills enhances quality of life more than peer support: results of a randomized trial with multiple sclerosis patients

    Health Psychol

    (1999)
  • G.S. Howard

    Response-shift bias: a source of contamination of self-report measures

    J Appl Psychol

    (1979)
  • D.A. Ruta et al.

    A new approach to the measurement of quality of life. The patient-generated index

    Med Care

    (1994)
  • F.J. Oort

    Using structural equation modeling to detect response shifts and true change

    Qual Life Res

    (2005)
  • N.E. Mayo et al.

    Identifying response shift statistically at the individual level

    Qual Life Res

    (2008)
  • N.E. Mayo et al.

    Bridging the gap: the effectiveness of teaming a stroke coordinator with patient's personal physician on the outcome of stroke

    Age Ageing

    (2007)
  • S. Ahmed et al.

    Using the patient generated index to evaluate response shift post-stroke

    Qual Life Res

    (2005)
  • EuroQol Group

    EuroQol—a new facility for the measurement of health-related quality of life. The EuroQol Group

    Health Policy

    (1990)
  • Cited by (33)

    • Semantic primes theory may be helpful in designing questionnaires such as to prevent response shift

      2015, Journal of Clinical Epidemiology
      Citation Excerpt :

      Thus, there is a need to disentangle RS effect from the effects of the “standard influences” on HRQL [11]. Various methods have been used to detect RS [12–21], although there was in the last years a focus on methods based on structural equation modeling (SEM). Indeed, some recent works were helpful in showing how RS effect can affect observed scores or true attributes (i.e., HRQL itself) and how this effect can be modeled in explaining changes in observed scores or HRQL over time [15,22].

    • Response shift and oral health quality of life in dentine hypersensitivity

      2014, Dentine Hypersensitivity: Developing a Person-centred Approach to Oral Health
    • Nonlinear effects of elevated temperature on grapevine phenology

      2013, Agricultural and Forest Meteorology
      Citation Excerpt :

      We used piecewise regression (De los Riosa and Garcia, 2011; Freed and Cann, 2010) to characterise the relationship between temperature effect and sugar concentration in controls. Regressions were fitted to the data pooled across experiments to probe for general patterns whereas analysis of residuals (Berg and Grace, 2011; Ericsson et al., 2009; Freed and Cann, 2010; Mayo et al., 2009; Monzon et al., 2006; Sadras et al., 2003; Tang et al., 2011) was used to statistically assess the effect of water regime (experiment 1), source:sink ratio (experiment 2) and variety (experiment 3) on phenological responses to temperature. To verify that observed differences in phenology were attributable to thermal regimes, we plotted the trajectories of berry sugar concentration on chronological and thermal-time scales.

    View all citing articles on Scopus
    View full text