Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research

https://doi.org/10.1016/S0277-9536(99)00047-7Get rights and content

Abstract

The impact of health state changes on an individual's quality of life (QOL) has gained increased attention in social and medical clinical research. An emerging construct of relevance to this line of investigation is response shift phenomenon. This construct refers to the changes in internal standards, in values, or in the conceptualization of QOL which are catalyzed by health state changes. In an effort to stimulate research on response shift, we present methodological considerations and promising assessment approaches for measuring it in observational and interventional clinical research. We describe and evaluate individualized methods, preference-based methods, successive comparison methods, design approaches, statistical approaches and qualitative approaches. The hierarchical structure of the construct is also discussed, with particular emphasis on how it might be elucidated by empirical assessment which uses the proposed methods and approaches. It is also recommended that criterion measures of change be included in future studies of response shift.

Introduction

Research on social factors and health relies heavily on the measurement of perceived quality of life (QOL). It is founded, however, on assumptions about the stability of intra-individual standards which may not be valid. As people can vary, so can their values. This variability may reflect informative shifts in an individual's internal standards, in values and priorities, or in the conceptualization of perceived QOL, in addition to changes in actual health state. This `response shift' phenomenon is of fundamental importance to social and medical science.

As an illustration, the results of a psychosocial randomized trial (Schwartz, in press) will be described, the objective of which was to compare the effectiveness of a coping skills group intervention as compared to a peer telephone support intervention for people with multiple sclerosis. These results motivated a quest for methods which measure response shift. The group intervention consisted of a comprehensive package which included cognitive, behavioral and supportive techniques aimed at helping patients to accommodate to their progressive disease trajectory. The peer telephone support intervention involved less intensive contact, no in-person meetings and was non-directive. Given the chronic and non-fatal nature of this autoimmune disease, patients had a prognosis for a normal lifespan with periods of relapses and progressive disability (Lechtenberg, 1988).

Two-year follow-up data suggested that despite significant deterioration in neuropsychological performance, neurological disability and self-efficacy function, participants in the group intervention reported reduced psychosocial role limitations, decreased use of negative coping strategies and enhanced well-being. In contrast, peer support recipients did not report significant changes in psychosocial role performance, coping, or well-being. Thus, although both groups exhibited significant deterioration in clinical measures of function, the well-known disparity between patient-reported and clinical measures of function was largest in the coping skills group.

The apparent separation of physical functioning and psychological well-being may represent `response shift phenomenon.' Our working definition of response shift refers to a change in the meaning of one's self-evaluation of a target construct as a result of: (a) a change in the respondent's internal standards of measurement (i.e. scale recalibration); (b) a change in the respondent's values (i.e. the importance of component domains constituting the target construct) or (c) a redefinition of the target construct (i.e. reconceptualization). Indeed, qualitative interviews with participants revealed that they felt that meeting people in the group who were worse or better off than they were led them to change their internal standards of how badly off they themselves were. Additionally, they felt that the group intervention helped them to reconsider the goals that were important and feasible to them (i.e. changes in values), and to learn that it was possible to have a reasonable QOL even with a worse condition (i.e. reconceptualization of QOL). This example illustrates how a psychosocial intervention might teach response shift.

The concept of response shift has its foundation in research on educational training interventions (Howard et al., 1979a, Howard et al., 1979b, Howard et al., 1979c, Howard et al., 1981) and organizational change (Golembiewski et al., 1976). Whereas Howard and colleagues defined response shift in terms of changes in internal standards of measurement, Golembiewski et al. introduced the component of changes in conceptualization and internal standards. Investigators from the discipline of management sciences described a typology of change which distinguished alpha, beta and gamma shifts, referring to objective changes, change in internal standards and reconceptualizations, respectively (Armenakis, 1988). The working definition adds changes in values as another component that is relevant to a change in the meaning of one's self-evaluation. While changes in values were inherent in the Golembiewski description of reconceptualization, the working definition adopted in this paper includes this as a separate third component that is relevant to the change in the meaning of one's self-evaluation. Making it a distinct third aspect will thus highlight its importance and emphasizes the need to measure it carefully (Sprangers and Schwartz, 1999). Finally, this working definition avoids the use of the terms `beta' and `gamma' shift since these terms do not reveal their content. It would be preferable to use the more transparent terms of change in internal standards, change in value and reconceptualization.

Response shift is important to consider in treatment evaluations, especially insofar as it may serve to attenuate or to exaggerate estimates of treatment effects as patients adapt to treatment toxicities or disease progression over time. Methods which assess response shift will not only be necessary to reveal unbiased treatment effects, but will also be useful for examining the impact of illness over time. The purpose of the present work is to discuss existing methods or those which can be developed to address the different components of response shift. As will be noted below, these approaches operationalize more than one aspect of response shift, a measurement challenge which may reflect an inherent inter-connection between internal standards, values and conceptualization. This inter-connection will be discussed in greater depth after the methods have been presented. Our goal is to encourage and facilitate research on this construct.

Section snippets

Methods

Several of the methods which will be described are protocols we are suggesting for adapting existing tools so that they can assess one or more of the three aspects of response shift. Others are new methods developed by the authors and for which empirical substantiation is currently underway. Finally, we also describe approaches which would influence the design of new studies or the analysis of existing data sets. Including outcome tools, design approaches and statistical methods in the

Discussion

We have discussed a number of methods which could be adapted or further developed to investigate response shift. The more established methods (e.g. those originating from educational or organizational change) have been discussed more critically than the relatively new methods because empirical evidence is available on the former to highlight their drawbacks. Consequently, the discussion of the respective methods has been necessarily unbalanced. Because very few methods have been designed

Acknowledgements

We would like to acknowledge the invaluable contribution of the following social and medical scientists who participated in the Response Shift Workshop which was held in Boston, December 1996 and funded by Agency for Health Care Policy and Research (1 RO1 HSO8582-01A1) to CES and by a matched contribution from Frontier Science. These participants included: Achilles Armenakis, Ph.D.; Katy Benjamin, SM, MSW; Lawren Daltroy, Dr.PH; Susan Folkman, Ph.D.; Rick Gibbons, Ph.D.; Maureen Wilson

References (88)

  • C Bakker et al.

    Journal of Rheumatology

    (1995)
  • D Bar-On et al.

    Re-examining quality of life of hypertensives: a new self-structured measure

    American Journal of Hypertension

    (1993)
  • A Bedeian et al.

    The significance of congruence coefficients: a comment and statistical test

    Journal of Management

    (1988)
  • J.H Bray et al.

    Methodological considerations in the evaluation of a teacher-training program

    Journal of Educational Psychology

    (1980)
  • A.S Bryk et al.

    Application of hierarchical linear models to assessing change

    Psych. Bull.

    (1987)
  • J.P Browne et al.

    Individual quality of life in the healthy elderly

    Quality of Life Research

    (1994)
  • M Buckley et al.

    Detecting scale recalibration in survey research: a laboratory investigation

    Group and Organization Studies

    (1987)
  • K.C Calman

    Quality of life in cancer patients: an hypothesis

    Journal of Medical Ethics

    (1984)
  • H Cantril

    The Pattern of Human Concerns

    (1966)
  • R Carr-Hill

    The measurement of patient satisfaction

    Journal of Public Health Medicine

    (1992)
  • W.G Cochran et al.

    Experimental Designs

    (1957)
  • R Cross

    On the foundations of hysteresis in economic systems

    Economics and Philosophy

    (1993)
  • P.H Ditto et al.

    Fates worse than death: the role of valued life activities in health-state evaluations

    Health Psychology

    (1996)
  • A.L Edwards

    The method of paired comparisons

  • R.C Elston et al.

    Estimation of time response curves and their confidence bands

    Biometrics

    (1962)
  • D.J Francis et al.

    Analysis of change: modeling individual growth

  • M Friedlin et al.

    A threshold estimation problem for processes with hysteresis

    Statistics and Probability Letters

    (1998)
  • R.T Golembiewski et al.

    Measuring change and persistence in human affairs: types of change generated by OLD designs

    J Applied Behavioral Science

    (1976)
  • M.J Granier et al.

    An experimental approach to evaluate methods to detect scale recalibration

    Educational and Psychological Measurement

    (1991)
  • J Hoogstraten

    The retrospective pretest in an educational training context

    Journal of Experimental Education

    (1982)
  • J Hoogstraten

    Influence of objective measures on self-reports in a retrospective pretest–posttest design

    Journal of Experimental Education

    (1985)
  • G.S Howard et al.

    Response shift bias: a source of contamination of self-report measures

    Journal of Applied Psychology

    (1979)
  • G.S Howard et al.

    Internal invalidity in pretest–posttest self-report evaluations and a re-evaluation of retrospective pretests

    Applied Psychological Measurement

    (1979)
  • G.S Howard et al.

    The feasibility of informed pretests in attenuating response-shift bias

    Applied Psychological Measurement

    (1979)
  • G.S Howard et al.

    Influence of subject response style effects on retrospective measures

    Applied Psychological Measurement

    (1981)
  • G.S Howard et al.

    Internal invalidity in studies employing self-report instruments. A suggested remedy

    Journal of Educational Measurement

    (1979)
  • B Jambon et al.

    Individual quality of life and clinical trials

    Quality of Life Newsletter

    (1997)
  • K Jöreskog et al.

    LISREL V: Analysis of Linear Structural Relationships by Maximum Likelihood and Least Square Methods

    (1981)
  • D Kahneman et al.

    Prospect theory: an analysis of decision under risk

    Econometrica

    (1979)
  • B Korth et al.

    The distribution of chance congruence coefficients from simulated data

    Psychometrika

    (1975)
  • N.M Laird et al.

    Random-effects models for longitudinal data

    Biometrics

    (1982)
  • R Lechtenberg

    Multiple Sclerosis Fact Book

    (1988)
  • Lenert, L.A., Morss, S., Goldstein, M.K., Bergen, M.R., Faustman, W.D., Garber, A.M., 1997. Measurement of the validity...
  • W Levinson et al.

    Retrospective versus actual pre-course self-assessments

    Evaluation Health Profession

    (1990)
  • Cited by (650)

    View all citing articles on Scopus
    View full text