Skip to main content

Longitudinal Change in Response Processes: A Response Shift Perspective

  • Chapter
  • First Online:

Part of the book series: Social Indicators Research Series ((SINS,volume 69))

Abstract

This chapter focuses on the examination of response shift in patient-reported outcomes (PROs) research, with particular attention to measurement validity and response processes. Response shift occurs when changes in PROs over time are the result of changes in how people interpret and respond to PRO measurement items at different points in time. Conceptual foundations of response shift are discussed, followed by a review of statistical methods for examining response shift. The chapter concludes with a discussion of opportunities and challenges for response shift research. We specifically consider that, although response shift is inherently about response processes and measurement validation, most of the methods and applications of response shift are descriptive in nature with an eye towards detecting and controlling for response shift. There is significant opportunity for theoretical and methodological development in focusing on understanding the mechanisms (mediators, moderators and other causes) by which response shift occurs when measuring PROs.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., et al. (1993). The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85, 365–376.

    Article  Google Scholar 

  • Ahmed, S., Mayo, N. E., Wood-Dauphinee, S., Hanley, J. A., & Cohen, S. R. (2005). The structural equation modeling technique did not show a response shift, contrary to the results of the then test and the individualized approaches. Journal of Clinical Epidemiology, 58, 1125–1133. doi:10.1016/j.jclinepi.2005.03.003.

    Article  Google Scholar 

  • Ahmed, S., Sawatzky, R., Levesque, J. F., Ehrmann-Feldman, D., & Schwartz, C. E. (2014). Minimal evidence of response shift in the absence of a catalyst. Quality of Life Research, 23, 2421–2430. doi:10.1007/s11136-014-0699-3.

    Article  Google Scholar 

  • Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology, 37, 1–15.

    Article  Google Scholar 

  • Anota, A., Bascoul-Mollevi, C., Conroy, T., Guillemin, F., Velten, M., Jolly, D., et al. (2014). Item response theory and factor analysis as a mean to characterize occurrence of response shift in a longitudinal quality of life study in breast cancer patients. Health and Quality of Life Outcomes, 12, 32. doi:10.1186/1477-7525-12-32.

    Article  Google Scholar 

  • Barclay-Goddard, R., Lix, L. M., Tate, R., Weinberg, L., & Mayo, N. E. (2009). Response shift was identified over multiple occasions with a structural equation modeling framework. Journal of Clinical Epidemiology, 62, 1181–1188. doi:10.1016/j.jclinepi.2009.03.014.

    Article  Google Scholar 

  • Beaumont, J. L., Lix, L. M., Yost, K. J., & Hahn, E. A. (2006). Application of robust statistical methods for sensitivity analysis of health-related quality of life outcomes. Quality of Life Research, 15, 349–356. doi:10.1007/s11136-005-2293-1.

    Article  Google Scholar 

  • Beeken, R., Eiser, C., & Sheeran, P. (2008). Response shift and Quality of Life (QOL): A priming study. Psychology & Health, 23, 63. doi:10.1007/s11136-010-9737-y.

    Google Scholar 

  • Bernhard, J., Lowy, A., Maibach, R., & Hurny, C. (2001). Response shift in the perception of health for utility evaluation. An explorative investigation. European Journal of Cancer, 37, 1729–1735.

    Google Scholar 

  • Boucekine, M., Loundou, A., Baumstarck, K., Minaya-Flores, P., Pelletier, J., Ghattas, B., & Auquier, P. (2013). Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: A cohort study. BMC Medical Research Methodology, 13, 20. doi:10.1186/1471-2288-13-20.

    Article  Google Scholar 

  • Bray, J. H., Maxwell, S. E., & Howard, G. S. (1984). Methods of analysis with response-shift bias. Educational and Psychological Measurement, 44, 781–804.

    Article  Google Scholar 

  • Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32.

    Article  Google Scholar 

  • Carroll, R. J., Ruppert, D., Stefanski, L. A., & Crainiceanu, C. M. (2006). Measurement error in nonlinear models: A modern perspective (2nd ed.). Boca Raton, FL: Chapman & Hall.

    Book  Google Scholar 

  • Cohen, S. R., Mount, B. M., Tomas, J. J., & Mount, L. F. (1996). Existential well-being is an important determinant of quality of life: Evidence from the McGill Quality of Life Questionnaire. Cancer, 77, 576–586.

    Article  Google Scholar 

  • Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. I. Braun (Eds.), Validity (pp. 3–17). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. doi:10.1037/h0040957.

    Article  Google Scholar 

  • Department of Health. (2009). Guidance on the routine collection of Patient Reported Outcomes Measures (PROMs). Retrieved from http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_092647.

  • Fayers, P., & Machin, D. (2007). Quality of life: The assessment, analysis and interpretation of patient-reported outcomes. Chichester, UK: John Wiley & Sons.

    Book  Google Scholar 

  • Guilleux, A., Blanchin, M., Vanier, A., Guillemin, F., Falissard, B., Schwartz, C. E., et al. (2015). RespOnse Shift ALgorithm in Item response theory (ROSALI) for response shift detection with missing data in longitudinal patient-reported outcome studies. Quality of Life Research, 24, 553–564. doi:10.1007/s11136-014-0876-4.

    Article  Google Scholar 

  • Hickey, A. M., Bury, G., O’Boyle, C. A., Bradley, F., O’Kelly, F. D., & Shannon, W. (1996). A new short form individual quality of life measure (SEIQoL-DW): Application in a cohort of individuals with HIV/AIDS. British Medical Journal, 313, 29–33.

    Article  Google Scholar 

  • Huberty, C. J., & Wisenbaker, J. M. (1992). Variable importance in multivariate group comparisons. Journal of Educational and Behavioral Statistics, 17, 75–91.

    Article  Google Scholar 

  • Hubley, A. M., & Zumbo, B. D. (1996). A dialectic on validity: Where we have been and where we are going. Journal of General Psychology, 123, 207–215.

    Article  Google Scholar 

  • Hubley, A. M., & Zumbo, B. D. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103, 219–230. doi:10.1007/s11205-011-9843-4.

    Article  Google Scholar 

  • Hubley, A. M., & Zumbo, B. D. (2013). Psychometric characteristics of assessment procedures: An overview. In K. F. Geisinger, B. A. Bracken, J. F. Carlson, J. I. C. Hansen, N. R. Kuncel, S. P. Reise, & M. C. Rodriguez (Eds.), APA handbook of testing and assessment in psychology, Vol. 1: Test theory and testing and assessment in industrial and organizational psychology (pp. 3–19). Washington, DC: American Psychological Association.

    Google Scholar 

  • Jobe, J. B. (2003). Cognitive psychology and self-reports: Models and methods. Quality of Life Research, 12, 219–227.

    Article  Google Scholar 

  • Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of EducationalMeasurement, 50, 1–73. doi:10.1111/jedm.12000.

    Google Scholar 

  • Kim, S., & Camilli, G. (2014). An item response theory approach to longitudinal analysis with application to summer setback in preschool language/literacy. Large-scale Assessments in Education, 2, 1–17. doi:10.1186/2196-0739-2-1.

    Google Scholar 

  • King-Kallimanis, B. L., Oort, F. J., & Garst, G. J. A. (2010). Using structural equation modelling to detect measurement bias and response shift in longitudinal data. AStA Advances in Statistical Analysis, 94, 139–156. doi:10.1007/s10182-010-0129-y.

    Article  Google Scholar 

  • King-Kallimanis, B. L., Oort, F. J., Nolte, S., Schwartz, C. E., & Sprangers, M. A. (2011). Using structural equation modeling to detect response shift in performance and health-related quality of life scores of multiple sclerosis patients. Quality of Life Research, 20, 1527–1540. doi:10.1007/s11136-010-9844-9.

    Article  Google Scholar 

  • King-Kallimanis, B. L., Oort, F. J., Visser, M. R., & Sprangers, M. A. (2009). Structural equation modeling of health-related quality-of-life data illustrates the measurement and conceptual perspectives on response shift. Journal of Clinical Epidemiology, 62, 1157–1164. doi:10.1016/j.jclinepi.2009.04.004.

    Article  Google Scholar 

  • Li, Y., & Rapkin, B. (2009). Classification and regression tree uncovered hierarchy of psychosocial determinants underlying quality-of-life response shift in HIV/AIDS. Journal of Clinical Epidemiology, 62, 1138–1147. doi:10.1016/j.jclinepi.2009.03.021.

    Article  Google Scholar 

  • Lix, L. M., Chan, E. K. H., Sawatzky, R., Sajobi, T. T., Liu, J., Hopman, W., & Mayo, N. (2015). Response shift and disease activity in inflammatory bowel disease. Quality of Life Research, 25, 1751–1760. doi:10.1007/s11136-015-1188-z.

    Article  Google Scholar 

  • Lix, L. M., Sajobi, T. T., Sawatzky, R., Liu, J., Mayo, N. E., Huang, Y., et al. (2013). Relative importance measures for reprioritization response shift. Quality of Life Research, 22, 695–703. doi:10.1007/s11136-012-0198-3.

    Article  Google Scholar 

  • Lowy, A., & Bernhard, J. (2004). Quantitative assessment of changes in patients’ constructs of quality of life: An application of multilevel models. Quality of Life Research, 13, 1177–1185.

    Article  Google Scholar 

  • Martin, F., Camfield, L., Rodham, K., Kliempt, P., & Ruta, D. (2007). Twelve years’ experience with the Patient Generated Index (PGI) of quality of life: A graded structured review. Quality of Life Research, 16, 705–715. doi:10.1007/s11136-006-9152-6.

    Article  Google Scholar 

  • Mayo, N. E., Scott, S. C., Dendukuri, N., Ahmed, S., & Wood-Dauphinee, S. (2008). Identifying response shift statistically at the individual level. Quality of Life Research, 17, 627–639. doi:10.1007/s11136-008-9329-2.

    Article  Google Scholar 

  • McClimans, L., Bickenbach, J., Westerman, M., Carlson, L., Wasserman, D., & Schwartz, C. (2013). Philosophical perspectives on response shift. Quality of Life Research, 22, 1871–1878. doi:10.1007/s11136-012-0300-x.

    Article  Google Scholar 

  • Meade, A. W., Lautenschlager, G. J., & Hecht, J. E. (2005). Establishing measurement equivalence and invariance in longitudinal data with item response theory. International Journal of Testing, 5, 279–300. doi:10.1207/s15327574ijt0503_6.

    Article  Google Scholar 

  • Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: Macmillan Publishing Co Inc..

    Google Scholar 

  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.

    Article  Google Scholar 

  • Messick, S. (1998). Test validity: A matter of consequence. Social Indicators Research, 45, 35–44.

    Article  Google Scholar 

  • Millsap, R. E. (2010). Testing measurement invariance using item response theory in longitudinal data: An introduction. Child Development Perspectives, 4, 5–9. doi:10.1111/j.1750-8606.2009.00109.x.

    Article  Google Scholar 

  • Neuman, H. B., Park, J., Fuzesi, S., & Temple, L. K. (2012). Rectal cancer patients’ quality of life with a temporary stoma: Shifting perspectives. Diseases of the Colon and Rectum, 55, 1117–1124. doi:10.1097/DCR.0b013e3182686213.

    Article  Google Scholar 

  • Nolte, S., Elsworth, G. R., Sinclair, A. J., & Osborne, R. H. (2009). Tests of measurement invariance failed to support the application of the “then-test”. Journal of Clinical Epidemiology, 62, 1173–1180. doi:10.1016/j.jclinepi.2009.01.021.

    Article  Google Scholar 

  • Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.

    Article  Google Scholar 

  • Oort, F. J., Visser, M. R., & Sprangers, M. A. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609.

    Article  Google Scholar 

  • Oort, F. J., Visser, M. R., & Sprangers, M. A. (2009). Formal definitions of measurement bias and explanation bias clarify measurement and conceptual perspectives on response shift. Journal of Clinical Epidemiology, 62, 1126–1137. doi:10.1016/j.jclinepi.2009.03.013.

    Article  Google Scholar 

  • Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2, 14. doi:10.1186/1477-7525-2-14.

    Article  Google Scholar 

  • Rapkin, B. D., & Schwartz, C. E. (2016). Distilling the essence of appraisal: A mixed methods study of people with multiple sclerosis. Quality of Life Research, 25, 793–805.

    Google Scholar 

  • Ruta, D. A., Garratt, A. M., Leng, M., Russell, I. T., & MacDonald, L. M. (1994). A new approach to the measurement of quality of life. The Patient-Generated Index. Medical Care, 32, 1109–1126.

    Article  Google Scholar 

  • Sajobi, T. T., Fiest, K. M., & Wiebe, S. (2014). Changes in quality of life after epilepsy surgery: The role of reprioritization response shift. Epilepsia, 55, 1331–1338. doi:10.1111/epi.12697.

    Article  Google Scholar 

  • Sajobi, T. T., Lix, L. M., Clara, I., Walker, J., Graff, L. A., Rawsthorne, P., et al. (2012). Measures of relative importance for health-related quality of life. Quality of Life Research, 21, 1–11. doi:10.1007/s11136-011-9914-7.

    Article  Google Scholar 

  • Sajobi, T. T., Lix, L. M., Singh, G., Lowerison, M., Engbers, J., & Mayo, N. E. (2015). Identifying reprioritization response shift in a stroke caregiver population: A comparison of missing data methods. Quality of Life Research, 24, 529–540. doi:10.1007/s11136-014-0824-3.

    Article  Google Scholar 

  • Samejima, F. (1997). Graded response model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 85–100). New York, NY: Springer.

    Chapter  Google Scholar 

  • Sawatzky, R., Chan, E. K. H., Zumbo, B. D., Bingham, C. O., Gardner, W., Jutai, J.,… Lix, L. M. (in press). Challenges and opportunities in patient-reported outcomes validation. Journal of Clinical Epidemiology. doi:10.1016/j.jclinepi.2016.12.002

  • Sawatzky, R., Gadermann, A., Ratner, P. A., Zumbo, B. D., & Lix, L. M. (2012). Identifying individuals with inflammatory bowel disease who experienced response shift: A latent class analysis. Quality of Life Research, 21, 33.

    Article  Google Scholar 

  • Schmitt, N. (1982). The use of analysis of covariance structures to assess beta and gamma change. Multivariate Behavioral Research, 17, 343–358.

    Article  Google Scholar 

  • Schmitt, N., Pulakos, E., & Lieblein, A. (1984). A comparison of three techniques to assess group-level beta and gamma change. Applied Psychological Measurement, 8, 249–260.

    Article  Google Scholar 

  • Schwartz, C. E., Ahmed, S., Sawatzky, R., Sajobi, T., Mayo, N., Finkelstein, J., et al. (2013). Guidelines for secondary analysis in search of response shift. Quality of Life Research, 22, 2663–2673. doi:10.1007/s11136-013-0402-0.

    Article  Google Scholar 

  • Schwartz, C. E., Bode, R., Repucci, N., Becker, J., Sprangers, M. A., & Fayers, P. M. (2006). The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research, 15, 1533–1550. doi:10.1007/s11136-006-0025-9.

    Article  Google Scholar 

  • Schwartz, C. E., & Rapkin, B. A. (2012). Understanding appraisal processes underlying the thentest: A mixed methods investigation. Quality of Life Research, 21, 381–388. doi:10.1007/s11136-011-0023-4.

    Article  Google Scholar 

  • Schwartz, C. E., Sajobi, T. T., Verdam, M. G., Sebille, V., Lix, L. M., Guilleux, A., & Sprangers, M. A. (2015). Method variation in the impact of missing data on response shift detection. Quality of Life Research, 24, 521–528. doi:10.1007/s11136-014-0746-0.

    Article  Google Scholar 

  • Schwartz, C. E., & Sprangers, M. A. (2009). Reflections on genes and sustainable change: Toward a trait and state conceptualization of response shift. Journal of Clinical Epidemiology, 62, 1118–1123. doi:10.1016/j.jclinepi.2009.02.008.

    Article  Google Scholar 

  • Schwartz, C. E., & Sprangers, M. A. (2010). Guidelines for improving the stringency of response shift research using the thentest. Quality of Life Research, 19, 455–464. doi:10.1007/s11136-010-9585-9.

    Article  Google Scholar 

  • Sprangers, M. A., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48, 1507–1515.

    Article  Google Scholar 

  • Thomas, D. R. (1992). Interpreting discriminant functions: A data analytic approach. Multivariate Behavioral Research, 27, 335–362. doi:10.1207/s15327906mbr2703_3.

    Article  Google Scholar 

  • Thomas, D. R., Hughes, E., & Zumbo, B. D. (1998). On variable importance in linear regression. Social Indicators Research, 45, 253–275.

    Article  Google Scholar 

  • Thomas, D. R., Zhu, P., Zumbo, B. D., & Dutta, S. (2008). On measuring the relative importance of explanatory variables in a logistic regression. Journal of Modern Applied Statistical Methods, 7, 21–38.

    Google Scholar 

  • Thomas, D. R., & Zumbo, B. D. (1996). Using a measure of variable importance to investigate the standardization of discriminant coefficients. Journal of Educational and Behavioral Statistics, 21, 110–130. doi:10.3102/10769986021002110.

    Article  Google Scholar 

  • Tourangeau, R., Rips, L. J., & Rasinski, K. A. (2000). The psychology of survey response. Cambridge, U.K.: Cambridge University Press.

    Book  Google Scholar 

  • Verdam, M. G., Oort, F. J., & Sprangers, M. A. G. (2016). Using structural equation modeling to detect response shifts and true change in discrete variables: An application to the items of the SF-36. Quality of Life Research, 25, 1361–1383. doi:10.1007/s11136-015-1195-0.

    Article  Google Scholar 

  • Verdam, M. G., Oort, F. J., van der Linden, Y. M., & Sprangers, M. A. (2015). Taking into account the impact of attrition on the assessment of response shift and true change: A multigroup structural equation modeling approach. Quality of Life Research, 24, 541–551. doi:10.1007/s11136-014-0829-y.

    Article  Google Scholar 

  • Westerman, M. J., Hak, T., Sprangers, M. A., Groen, H. J., van der Wal, G., & The, A. M. (2008). Listen to their answers! Response behaviour in the measurement of physical and role functioning. Quality of Life Research, 17, 549–558. doi:10.1007/s11136-008-9333-6.

    Article  Google Scholar 

  • Zumbo, B. D. (2007a). Validity: Foundational issues and statistical methodology. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics (Vol. 26: Psychometrics, pp. 45–79). Amsterdam, the Netherlands: Elsevier Science.

    Google Scholar 

  • Zumbo, B. D. (2007b). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4, 223–233.

    Article  Google Scholar 

  • Zumbo, B. D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions and applications (pp. 65–82). Charlotte, NC: Information Age Publishing.

    Google Scholar 

  • Zumbo, B. D., & Chan, E. K. H. (2014). Reflections on validation practices in the social, behavioral, and health sciences. In B. D. Zumbo & E. K. H. Chan (Eds.), Validity and validation in social, behavioral, and health sciences (Vol. 54, pp. 321–327). New York, NY: Springer International Publishing.

    Google Scholar 

Download references

Acknowledgements

Writing of this chapter was supported by a CIHR Operating grant (#342467) held by RS, TTS, LML, and BZ; a Research Manitoba operating grant held by LM; an O’Brien Institute for Public Health Catalyst award held by TTS; Canada Research Chairs program funding for a Canada Research Chair in Patient-Reported Outcomes held by RS; a Manitoba Research Chair held by LL.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Richard Sawatzky .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Sawatzky, R., Sajobi, T.T., Brahmbhatt, R., Chan, E.K.H., Lix, L.M., Zumbo, B.D. (2017). Longitudinal Change in Response Processes: A Response Shift Perspective. In: Zumbo, B., Hubley, A. (eds) Understanding and Investigating Response Processes in Validation Research. Social Indicators Research Series, vol 69. Springer, Cham. https://doi.org/10.1007/978-3-319-56129-5_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-56129-5_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-56128-8

  • Online ISBN: 978-3-319-56129-5

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics