Skip to main content
Top
Gepubliceerd in:

27-10-2021

If it’s information, it’s not “bias”: a scoping review and proposed nomenclature for future response-shift research

Auteurs: Carolyn E. Schwartz, Gudrun Rohde, Elijah Biletch, Richard B. B. Stuart, I.-Chan Huang, Joseph Lipscomb, Roland B. Stark, Richard L. Skolasky

Gepubliceerd in: Quality of Life Research | Uitgave 8/2022

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Background

The growth in response-shift methods has enabled a stronger empirical foundation to investigate response-shift phenomena in quality-of-life (QOL) research; but many of these methods utilize certain language in framing the research question(s) and interpreting results that treats response-shift effects as “bias,” “noise,” “nuisance,” or otherwise warranting removal from the results rather than as information that matters. The present project will describe the various ways in which researchers have framed the questions for investigating response-shift issues and interpreted the findings, and will develop a nomenclature for such that highlights the important information about resilience reflected by response-shift findings.

Methods

A scoping review was done of the QOL and response-shift literature (n = 1100 articles) from 1963 to 2020. After culling only empirical response-shift articles, raters characterized how investigators framed and interpreted study research questions (n = 164 articles).

Results

Of 10 methods used, papers using four of them utilized terms like “bias” and aimed to remove response-shift effects to reveal “true change.” Yet, the investigators’ reflections on their own conclusions suggested that they do not truly believe that response shift is error to be removed. A structured nomenclature is proposed for discussing response-shift results in a range of research contexts and response-shift detection methods.

Conclusions

It is time for a concerted and focused effort to change the nomenclature of those methods that demonstrated this misinterpretation. Only by framing and interpreting response shift as information, not bias, can we improve our understanding and methods to help to distill outcomes with and without response-shift effects.
Bijlagen
Alleen toegankelijk voor geautoriseerde gebruikers
Voetnoten
1
From Shakespeare’s Hamlet, 1602 [34], paraphrased (‘Though this be madness yet there is method in it’).
 
2
These articles did not mention the response shift phenomenon as we define it. For example, some early papers on visual stimulation used the term ‘response shift’ in a different way.
 
3
With SEM, we define response shift as a violation of conditional independence, i.e., a violation of f(X|T) = f(X|T,V). In the SEM method T is operationalized as a latent construct measured with a set of indicator variables X. If most of the X are affected by response shift, then we cannot distinguish response shift from T."
 
Literatuur
1.
go back to reference Sprangers, M. A. G., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48(11), 1507–1515.CrossRef Sprangers, M. A. G., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48(11), 1507–1515.CrossRef
2.
go back to reference Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2(1), 14.CrossRef Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2(1), 14.CrossRef
4.
go back to reference Schwartz, C. E., & Rapkin, B. D. (2004). Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal. Health and Quality of Life Outcomes, 2, 16.CrossRef Schwartz, C. E., & Rapkin, B. D. (2004). Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal. Health and Quality of Life Outcomes, 2, 16.CrossRef
5.
go back to reference Bach, J. R., & Tilton, M. C. (1994). Life satisfaction and well-being measures in ventilator assisted individuals with traumatic tetraplegia. Archives of Physical Medicine and Rehabilitation, 75(6), 626–632.CrossRef Bach, J. R., & Tilton, M. C. (1994). Life satisfaction and well-being measures in ventilator assisted individuals with traumatic tetraplegia. Archives of Physical Medicine and Rehabilitation, 75(6), 626–632.CrossRef
8.
go back to reference Dobkin, B. H., Nadeau, S. E., Behrman, A. L., Wu, S. S., Rose, D. K., Bowden, M., et al. (2014). Prediction of responders for outcome measures of locomotor experience applied post stroke trial. Journal of Rehabilitation Research and Development, 51(1), 39.CrossRef Dobkin, B. H., Nadeau, S. E., Behrman, A. L., Wu, S. S., Rose, D. K., Bowden, M., et al. (2014). Prediction of responders for outcome measures of locomotor experience applied post stroke trial. Journal of Rehabilitation Research and Development, 51(1), 39.CrossRef
14.
go back to reference Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.CrossRef Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.CrossRef
15.
go back to reference Oort, F. J., Visser, M. R. M., & Sprangers, M. A. G. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609.CrossRef Oort, F. J., Visser, M. R. M., & Sprangers, M. A. G. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609.CrossRef
17.
go back to reference Sajobi, T. T., Fiest, K. M., & Wiebe, S. (2014). Changes in quality of life after epilepsy surgery: the role of reprioritization response shift. Epilepsia, 55, 1331–1338.CrossRef Sajobi, T. T., Fiest, K. M., & Wiebe, S. (2014). Changes in quality of life after epilepsy surgery: the role of reprioritization response shift. Epilepsia, 55, 1331–1338.CrossRef
18.
go back to reference Sawatzky, R., Gadermann, A. M., Ratner, P. A., Zumbo, B., & Lix, L. (2012). Identifying individuals with inflammatory bowel disease who experienced response shift: A latent class analysis. Quality of Life Research, 21, 33.CrossRef Sawatzky, R., Gadermann, A. M., Ratner, P. A., Zumbo, B., & Lix, L. (2012). Identifying individuals with inflammatory bowel disease who experienced response shift: A latent class analysis. Quality of Life Research, 21, 33.CrossRef
19.
go back to reference Guilleux, A., Blanchin, M., Vanier, A., Guillemin, F., Falissard, B., Schwartz, C. E., et al. (2015). Response shift algorithm in item response theory (ROSALI) for response shift detection with missing data in patient-reported outcomes in longitudinal clinical trials. Quality of Life Research, 24(3), 553–564.CrossRef Guilleux, A., Blanchin, M., Vanier, A., Guillemin, F., Falissard, B., Schwartz, C. E., et al. (2015). Response shift algorithm in item response theory (ROSALI) for response shift detection with missing data in patient-reported outcomes in longitudinal clinical trials. Quality of Life Research, 24(3), 553–564.CrossRef
21.
go back to reference Sajobi, T. T., Brahmbatt, R., Lix, L. M., Zumbo, B. D., & Sawatzky, R. (2018). Scoping review of response shift methods: Current reporting practices and recommendations. Quality of Life Research, 27(5), 1133–1146.CrossRef Sajobi, T. T., Brahmbatt, R., Lix, L. M., Zumbo, B. D., & Sawatzky, R. (2018). Scoping review of response shift methods: Current reporting practices and recommendations. Quality of Life Research, 27(5), 1133–1146.CrossRef
24.
go back to reference Nunnally, J. C. (1994). Psychometric Theory (3rd ed.). Tata McGraw-Hill Education. Nunnally, J. C. (1994). Psychometric Theory (3rd ed.). Tata McGraw-Hill Education.
25.
go back to reference Edelen, M. O., Thissen, D., Teresi, J. A., Kleinman, M., & Ocepek-Welikson, K. (2006). Identification of differential item functioning using item response theory and the likelihood-based model comparison approach: application to the mini-mental state examination. Medical Care, 44, S134–S142.CrossRef Edelen, M. O., Thissen, D., Teresi, J. A., Kleinman, M., & Ocepek-Welikson, K. (2006). Identification of differential item functioning using item response theory and the likelihood-based model comparison approach: application to the mini-mental state examination. Medical Care, 44, S134–S142.CrossRef
27.
go back to reference Oort, F. J. (2007) Bias is not a negatively valenced term In Personal Communication (Ed) Boston MA Oort, F. J. (2007) Bias is not a negatively valenced term In Personal Communication (Ed) Boston MA
28.
go back to reference Delgado-Rodriguez, M., & Llorca, J. (2004). Bias. Journal of Epidemiology & Community Health, 58(8), 635–641.CrossRef Delgado-Rodriguez, M., & Llorca, J. (2004). Bias. Journal of Epidemiology & Community Health, 58(8), 635–641.CrossRef
29.
go back to reference Sapir, E. (1929). The status of linguistics as a science. Language, 5(4), 207–214.CrossRef Sapir, E. (1929). The status of linguistics as a science. Language, 5(4), 207–214.CrossRef
30.
go back to reference Whorf, B. L. (1940). Science and Linguistics: Bobbs-Merrill Indianapolis. IN. Whorf, B. L. (1940). Science and Linguistics: Bobbs-Merrill Indianapolis. IN.
32.
go back to reference Martinez, J. E., Feldman, L. A., Feldman, M. J., & Cikara, M. (2021). Narratives shape cognitive representations of immigrants and immigration-policy preferences. Psychological Science, 32(2), 135–152.CrossRef Martinez, J. E., Feldman, L. A., Feldman, M. J., & Cikara, M. (2021). Narratives shape cognitive representations of immigrants and immigration-policy preferences. Psychological Science, 32(2), 135–152.CrossRef
33.
go back to reference Munn, Z., Peters, M. D., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1), 1–7.CrossRef Munn, Z., Peters, M. D., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1), 1–7.CrossRef
34.
go back to reference Shakespeare, W., Evans, G. B., Levin, H., & Shattuck, C. H. (1974). The Riverside Shakespeare. In (Vol. 1, pp. 1140): Houghton Mifflin Boston. Shakespeare, W., Evans, G. B., Levin, H., & Shattuck, C. H. (1974). The Riverside Shakespeare. In (Vol. 1, pp. 1140): Houghton Mifflin Boston.
35.
go back to reference Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, D. W., & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Applied Psychol Meas, 3(1), 1–23.CrossRef Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, D. W., & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Applied Psychol Meas, 3(1), 1–23.CrossRef
36.
go back to reference Wilson, I. B. (1999). Clinical understanding and clinical implications of response shift. Social Science and Medicine, 48(11), 1577–1588.CrossRef Wilson, I. B. (1999). Clinical understanding and clinical implications of response shift. Social Science and Medicine, 48(11), 1577–1588.CrossRef
37.
go back to reference Schwartz, C. E., & Sendor, M. (2000). Helping others helps oneself: Response shift effects in peer support. In C. E. Schwartz & M. A. G. Sprangers (Eds.), Adaptation to changing health: Response shift in quality-of-life research (pp. 175–188). American Psychological Association.CrossRef Schwartz, C. E., & Sendor, M. (2000). Helping others helps oneself: Response shift effects in peer support. In C. E. Schwartz & M. A. G. Sprangers (Eds.), Adaptation to changing health: Response shift in quality-of-life research (pp. 175–188). American Psychological Association.CrossRef
40.
go back to reference Boucekine, M., Loundou, A., Baumstarck, K., Minaya-Flores, P., Pelletier, J., Ghattas, B., et al. (2013). Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: a cohort study. BMC Medical Research Methodology, 13, 20.CrossRef Boucekine, M., Loundou, A., Baumstarck, K., Minaya-Flores, P., Pelletier, J., Ghattas, B., et al. (2013). Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: a cohort study. BMC Medical Research Methodology, 13, 20.CrossRef
41.
go back to reference O’Boyle, C. A. (1994). The schedule for the evaluation of individual quality of life (SEIQoL). International Journal of Mental Health, 23(3), 3–23.CrossRef O’Boyle, C. A. (1994). The schedule for the evaluation of individual quality of life (SEIQoL). International Journal of Mental Health, 23(3), 3–23.CrossRef
42.
go back to reference Ruta, D. A., Garratt, A. M., Leng, M., Russell, I. T., & MacDonald, L. M. (1994). A new approach to the measurement of quality of life. The Patient-Generated Index. Med Care, 32(11), 1109–1126.PubMed Ruta, D. A., Garratt, A. M., Leng, M., Russell, I. T., & MacDonald, L. M. (1994). A new approach to the measurement of quality of life. The Patient-Generated Index. Med Care, 32(11), 1109–1126.PubMed
46.
go back to reference Schwartz, C. E., Sajobi, T. T., Lix, L. M., Quaranto, B. R., & Finkelstein, J. A. (2013). Changing values, changing outcomes: The influence of reprioritization response shift on outcome assessment after spine surgery. Quality of Life Research, 22(9), 2255–2264.CrossRef Schwartz, C. E., Sajobi, T. T., Lix, L. M., Quaranto, B. R., & Finkelstein, J. A. (2013). Changing values, changing outcomes: The influence of reprioritization response shift on outcome assessment after spine surgery. Quality of Life Research, 22(9), 2255–2264.CrossRef
49.
go back to reference Schwartz, C. E., Stucky, B. D., Michael, W., & Rapkin, B. D. (2020). Does response shift impact interpretation of change even among scales developed using item response theory? Journal of Patient-Reported Outcomes, 4(1), 8.CrossRef Schwartz, C. E., Stucky, B. D., Michael, W., & Rapkin, B. D. (2020). Does response shift impact interpretation of change even among scales developed using item response theory? Journal of Patient-Reported Outcomes, 4(1), 8.CrossRef
53.
go back to reference Aburub, A. S., Gagnon, B., Ahmed, S., Rodríguez, A. M., & Mayo, N. E. (2018). Impact of reconceptualization response shift on rating of quality of life over time among people with advanced cancer. Supportive Care in Cancer, 26, 3063–3071.CrossRef Aburub, A. S., Gagnon, B., Ahmed, S., Rodríguez, A. M., & Mayo, N. E. (2018). Impact of reconceptualization response shift on rating of quality of life over time among people with advanced cancer. Supportive Care in Cancer, 26, 3063–3071.CrossRef
54.
go back to reference Carlier, I. V. E., van Eeden, W. A., de Jong, K., Giltay, E. J., van Noorden, M. S., van der Feltz-Cornelis, C., et al. (2019). Testing for response shift in treatment evaluation of change in self-reported psychopathology amongst secondary psychiatric care outpatients. International Journal of Methods in Psychiatric Research, 28, e1785. https://doi.org/10.1002/mpr.1785CrossRefPubMedPubMedCentral Carlier, I. V. E., van Eeden, W. A., de Jong, K., Giltay, E. J., van Noorden, M. S., van der Feltz-Cornelis, C., et al. (2019). Testing for response shift in treatment evaluation of change in self-reported psychopathology amongst secondary psychiatric care outpatients. International Journal of Methods in Psychiatric Research, 28, e1785. https://​doi.​org/​10.​1002/​mpr.​1785CrossRefPubMedPubMedCentral
56.
go back to reference Li, Y., & Schwartz, C. E. (2011). Data mining for response shift patterns in multiple sclerosis patients using recursive partitioning tree analysis. Quality of Life Research, 20, 1543–1553.CrossRef Li, Y., & Schwartz, C. E. (2011). Data mining for response shift patterns in multiple sclerosis patients using recursive partitioning tree analysis. Quality of Life Research, 20, 1543–1553.CrossRef
59.
go back to reference Schwartz, C. E., Sprangers, M. A. G., Carey, A., & Reed, G. (2004). Exploring response shift in longitudinal data. Psychology & Health, 19(1), 51–69.CrossRef Schwartz, C. E., Sprangers, M. A. G., Carey, A., & Reed, G. (2004). Exploring response shift in longitudinal data. Psychology & Health, 19(1), 51–69.CrossRef
61.
go back to reference Sprangers, M. A. G., Taminiau-Bloem, E. F., Van Zuuren, F. J., Koeneman, M. A., Visser, M. R. M., Tishelman, C., et al. (2015). Using a retrospective pretest instead of a conventional pretest is replacing biases: A qualitative study of cognitive processes underlying responses to thentest items. Quality of Life Research, 24, 10. https://doi.org/10.1007/s11136-015-1078-4LKCrossRef Sprangers, M. A. G., Taminiau-Bloem, E. F., Van Zuuren, F. J., Koeneman, M. A., Visser, M. R. M., Tishelman, C., et al. (2015). Using a retrospective pretest instead of a conventional pretest is replacing biases: A qualitative study of cognitive processes underlying responses to thentest items. Quality of Life Research, 24, 10. https://​doi.​org/​10.​1007/​s11136-015-1078-4LKCrossRef
62.
go back to reference Oort, F. J. (2021). SEM method will only correctly find response shift if it applies to the majority of the respondents (or all respondents), in a minority of the indicator variables. Personal Communication, . Amsterdam, The Netherlands. Oort, F. J. (2021). SEM method will only correctly find response shift if it applies to the majority of the respondents (or all respondents), in a minority of the indicator variables. Personal Communication, . Amsterdam, The Netherlands.
63.
go back to reference Kline, R. B. (2015). Principles and practice of structural equation modeling. Guilford Publications. Kline, R. B. (2015). Principles and practice of structural equation modeling. Guilford Publications.
64.
go back to reference Sajobi, T., Anota, A., Awosoga, O., Lawal, O., Oomen, C., Brobbey, A., et al. (2020). A meta-analysis of response shift effects in patient-reported outcomes studies [Abstract]. Quality of Life Research, 29(Suppl 1), S2–S3. Sajobi, T., Anota, A., Awosoga, O., Lawal, O., Oomen, C., Brobbey, A., et al. (2020). A meta-analysis of response shift effects in patient-reported outcomes studies [Abstract]. Quality of Life Research, 29(Suppl 1), S2–S3.
66.
go back to reference Krägeloh, C. U., Bergomi, C., Siegert, R. J., & Medvedev, O. N. (2018). Response shift after a mindfulness-based intervention: Measurement invariance testing of the Comprehensive Inventory of Mindfulness Experiences. Mindfulness, 9(1), 212–220.CrossRef Krägeloh, C. U., Bergomi, C., Siegert, R. J., & Medvedev, O. N. (2018). Response shift after a mindfulness-based intervention: Measurement invariance testing of the Comprehensive Inventory of Mindfulness Experiences. Mindfulness, 9(1), 212–220.CrossRef
67.
go back to reference Osborne, R. H., Hawkins, M., & Sprangers, M. A. (2006). Change of perspective: A measurable and desired outcome of chronic disease self-management intervention programs that violates the premise of preintervention/postintervention assessment. Arthritis Care Re, 55(3), 458–465.CrossRef Osborne, R. H., Hawkins, M., & Sprangers, M. A. (2006). Change of perspective: A measurable and desired outcome of chronic disease self-management intervention programs that violates the premise of preintervention/postintervention assessment. Arthritis Care Re, 55(3), 458–465.CrossRef
68.
go back to reference Schwartz, C. (2005). Decision making at the end of life: Shifting sands. Journal of the Royal Society of Medicine, 98(7), 297–298.CrossRef Schwartz, C. (2005). Decision making at the end of life: Shifting sands. Journal of the Royal Society of Medicine, 98(7), 297–298.CrossRef
69.
go back to reference Schwartz, C. E., Stark, R. B., & Stucky, B. D. (2020). Response-shift effects in neuromyelitis optica spectrum disorder: a secondary analysis of clinical trial data. Quality of Life Research, 30, 1–16. Schwartz, C. E., Stark, R. B., & Stucky, B. D. (2020). Response-shift effects in neuromyelitis optica spectrum disorder: a secondary analysis of clinical trial data. Quality of Life Research, 30, 1–16.
70.
go back to reference Schwartz, C. E., Stark, R. B., Stucky, B. D., Li, Y., & Rapkin, B. D. (2021). Response-shift effects in neuromyelitis optica spectrum disorder: estimating response-shift-adjusted scores using equating. Qual Life Res, 1–10. Schwartz, C. E., Stark, R. B., Stucky, B. D., Li, Y., & Rapkin, B. D. (2021). Response-shift effects in neuromyelitis optica spectrum disorder: estimating response-shift-adjusted scores using equating. Qual Life Res, 1–10.
Metagegevens
Titel
If it’s information, it’s not “bias”: a scoping review and proposed nomenclature for future response-shift research
Auteurs
Carolyn E. Schwartz
Gudrun Rohde
Elijah Biletch
Richard B. B. Stuart
I.-Chan Huang
Joseph Lipscomb
Roland B. Stark
Richard L. Skolasky
Publicatiedatum
27-10-2021
Uitgeverij
Springer International Publishing
Gepubliceerd in
Quality of Life Research / Uitgave 8/2022
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-021-03023-9

Andere artikelen Uitgave 8/2022

Quality of Life Research 8/2022 Naar de uitgave