Top

Quality of Life Research

Gepubliceerd in:

18-11-2022 | Special Section: Methodologies for Meaningful Change

The challenges inherent with anchor-based approaches to the interpretation of important change in clinical outcome assessments

Auteurs: Kathleen W. Wyrwich, Geoffrey R. Norman

Gepubliceerd in: Quality of Life Research | Uitgave 5/2023

Abstract

Purpose

Anchor-based methods are group-level approaches used to derive clinical outcome assessment (COA) interpretation thresholds of meaningful within-patient change over time for understanding impacts of disease and treatment. The methods explore the associations between change in the targeted concept of the COA measure and the concept measured by the external anchor(s), typically a global rating, chosen as easier to interpret than the COA measure. While they are valued for providing plausible interpretation thresholds, group-level anchor-based methods pose a number of inherent theoretical and methodological conundrums for interpreting individual-level change.

Methods

This investigation provides a critical appraisal of anchor-based methods for COA interpretation thresholds and details key biases in anchor-based methods that directly influences the magnitude of the interpretation threshold.

Results

Five important research issues inherent with the use of anchor-based methods deserve attention: (1) global estimates of change are consistently biased toward the present state; (2) the use of static current state global measures, while not subject to artifacts of recall, may exacerbate the problem of estimating clinically meaningful change; (3) the specific anchor assessment response(s) that identify the meaningful change group usually involves an arbitrary judgment; (4) the calculated interpretation thresholds are sensitive to the proportion of patients who have improved; and (5) examination of anchor-based regression methods reveals that the correlation between the COA change scores and the anchor has a direct linear relationship to the magnitude of the interpretation threshold derived using an anchor-based approach; stronger correlations yielding larger interpretation thresholds.

Conclusions

While anchor-based methods are recognized for their utility in deriving interpretation thresholds for COAs, attention to the biases associated with estimation of the threshold using these methods is needed to progress in the development of standard-setting methodologies for COAs.

vorige artikel Common methods of determining meaningful change in clinical practice: implications for precision patient-reported outcomes

volgende artikel How scoring limits the usability of minimal important differences (MIDs) as responder definition (RD): an exemplary demonstration using EORTC QLQ-C30 subscales

Patient-Focused Drug Development. (2018). Guidance 3 discussion document: Select, develop or modify fit-for-purpose clinical outcomes assessments. Retrieved March 26, 2021, from https://www.fda.gov/media/116277/download

Deyo, R. A., & Inui, T. S. (1984). Toward clinical applications of health status measures: Sensitivity of scales to clinically important changes. Health Services Research, 19(3), 275–289.PubMedPubMedCentral

Jaeschke, R., Singer, J., & Guyatt, G. H. (1989). Measurement of health status. Ascertaining the minimal clinically important difference. Controlled Clinical Trials, 10(4), 407–415. https://doi.org/10.1016/0197-2456(89)90005-6CrossRefPubMed

U.S. FDA. (2009). US department of health and human services food and drug administration guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims. Retrieved March 26, 2021, from https://www.fda.gov/media/77832/download

Patient-Focused Drug Development. (2019). Guidance 4 discussion document: Incorporating clinical outcome assessments into endpoints for regulatory decision making. Retrieved March 26, 2021, from https://www.fda.gov/media/132505/download

King, M. T., Dueck, A. C., & Revicki, D. A. (2019). Can methods developed for interpreting group-level patient-reported outcome data be applied to individual patient management? Medical Care, 57(Suppl 5 1), S38–S45. https://doi.org/10.1097/MLR.0000000000001111CrossRefPubMedPubMedCentral

Patient-Focused Drug Development. (2018). Guidance 3 discussion document: Select, develop or modify fit-for-purpose clinical outcomes assessments. Appendices. Retrieved March 26, 2021, from https://www.fda.gov/media/116281/download

Norman, G. R., Stratford, P., & Regehr, G. (1997). Methodological problems in the retrospective computation of responsiveness to change: The lesson of Cronbach. Journal of Clinical Epidemiology., 50(8), 869–879. https://doi.org/10.1016/s0895-4356(97)00097-8CrossRefPubMed

Guyatt, G. H., Norman, G. R., Juniper, E. F., & Griffith, L. E. (2002). A critical look at transition ratings. Journal of Clinical Epidemiology., 55(9), 900–908. https://doi.org/10.1016/s0895-4356(02)00435-3CrossRefPubMed

10.

Schmitt, J., & Di Fabio, R. P. (2005). The validity of prospective and retrospective global change criterion measures. Archives of Physical Medicine and Rehabilitation, 86(12), 2270–2276. https://doi.org/10.1016/j.apmr.2005.07.290CrossRefPubMed

11.

Metz, S. M., Wyrwich, K. W., Babu, A. N., Kroenke, K., Tierney, W. M., & Wolinsky, F. D. (2007). Validity of patient-reported health-related quality of life global ratings of change using structural equation modeling. Quality of Life Research, 16(7), 1193–1202. https://doi.org/10.1007/s11136-007-9225-1CrossRefPubMed

12.

Devji, T., Carrasco-Labra, A., Qasim, A., Phillips, M., Johnston, B. C., Devasenapathy, N., Zeraatkar, D., Bhatt, M., Jin, X., Brignardello-Petersen, R., Urquhart, O., Foroutan, F., Schandelmaier, S., Pardo-Hernandez, H., Vernooij, R. W., Huang, H., Rizwan, Y., Siemieniuk, R., Lytvyn, L., & Guyatt, G. H. (2020). Evaluating the credibility of anchor based estimates of minimal important differences for patient reported outcomes: Instrument development and reliability study. British Medical Journal, 369, m1714. https://doi.org/10.1136/bmj.m1714CrossRefPubMedPubMedCentral

13.

Ward, C. L., & Wilson, A. E. (2015). Implicit theories of change and stability moderate effects of subjective distance on the remembered self. Personality and Social Psychology Bulletin, 41(9), 1167–1179. https://doi.org/10.1177/0146167215591571CrossRefPubMed

14.

Ross, M. (1989). Relation of implicit theories to the construction of personal histories. Psychological Review, 96(2), 341–357. https://doi.org/10.1037/0033-295X.96.2.341CrossRef

15.

Schwartz, C. E., Andresen, E. M., Nosek, M. A., Krahn, G. L., RRTC Expert Panel on Health Status Measurement. (2007). Response shift theory: Important implications for measuring quality of life in people with disability. Archives of Physical Medicine and Rehabilitation, 88(4), 529–536. https://doi.org/10.1016/j.apmr.2006.12.032CrossRefPubMed

16.

Fayers, P. M., & Hays, R. D. (2014). Don’t middle your MIDs: Regression to the mean shrinks estimates of minimally important differences. Quality of Life Research, 23(1), 1–4. https://doi.org/10.1007/s11136-013-0443-4CrossRefPubMed

17.

de Vet, H. C., Terluin, B., Knol, D. L., Roorda, L. D., Mokkink, L. B., Ostelo, R. W., Hendriks, E. J., Bouter, L. M., & Terwee, C. B. (2010). Three ways to quantify uncertainty in individually applied “minimally important change” values. Journal of Clinical Epidemiology, 63(1), 37–45. https://doi.org/10.1016/j.jclinepi.2009.03.011CrossRefPubMed

18.

Turner-Bowker, D. M., Lamoureux, R. E., Stokes, J., Litcher-Kelly, L., Galipeau, N., Yaworsky, A., Solomon, J., & Shields, A. L. (2018). Informing a priori sample size estimation in qualitative concept elicitation interview studies for clinical outcome assessment instrument development. Value in Health, 21(7), 839–842. https://doi.org/10.1016/j.jval.2017.11.014CrossRefPubMed

19.

Staunton, H., Willgoss, T., Nelsen, L., Burbridge, C., Sully, K., Rofail, D., & Arbuckle, R. (2019). An overview of using qualitative techniques to explore and define estimates of clinically important change on clinical outcome assessments. J Patient Rep Outcomes, 3(1), 16. https://doi.org/10.1186/s41687-019-0100-yCrossRefPubMedPubMedCentral

20.

Juniper, E. F., Guyatt, G. H., Willan, A., & Griffith, L. E. (1994). Determining a minimal important change in a disease-specific quality of life questionnaire. Journal of Clinical Epidemiology, 47(1), 81–87. https://doi.org/10.1016/0895-4356(94)90036-1CrossRefPubMed

21.

King, M. T. (2011). A point of minimal important difference (MID): A critique of terminology and methods. Expert Review of Pharmacoeconomics & Outcomes Research, 11(2), 171–184. https://doi.org/10.1586/erp.11.9CrossRef

22.

Lavigne, J. V. (2016). Systematic review: Issues in measuring clinically meaningful change in self-reported chronic pediatric pain intensity. Journal of Pediatric Psychology, 41(7), 715–734. https://doi.org/10.1093/jpepsy/jsv161CrossRefPubMed

23.

Terluin, B., Eekhout, I., & Terwee, C. B. (2017). The anchor-based minimal important change, based on receiver operating characteristic analysis or predictive modeling, may need to be adjusted for the proportion of improved patients. Journal of Clinical Epidemiology, 83, 90–100. https://doi.org/10.1016/j.jclinepi.2016.12.015CrossRefPubMed

24.

Terluin, B., Eekhout, I., Terwee, C. B., & de Vet, H. C. (2015). Minimal important change (MIC) based on a predictive modeling approach was more precise than MIC based on ROC analysis. Journal of Clinical Epidemiology, 68(12), 1388–1396. https://doi.org/10.1016/j.jclinepi.2015.03.015CrossRefPubMed

25.

Hays, R. D., Farivar, S. S., & Liu, H. (2005). Approaches and recommendations for estimating minimally important differences for health-related quality of life measures. Chronic Obstructive Pulmonary Diseases, 2(1), 63–67. https://doi.org/10.1081/copd-200050663CrossRef

26.

Revicki, D., Hays, R. D., Cella, D., & Sloan, J. (2008). Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Journal of Clinical Epidemiology, 61(2), 102–109. https://doi.org/10.1016/j.jclinepi.2007.03.012CrossRefPubMed

27.

Coon, C. D., & Cappelleri, J. C. (2016). Interpreting change in scores on patient-reported outcome instruments. Therapeutic Innovation and Regulatory Science, 50(1), 22–29. https://doi.org/10.1177/2168479015622667CrossRefPubMed

28.

Fayers, P. M., & Hays, R. D. (2014). Should linking replace regression when mapping from profile-based measures to preference-based measures? Value in Health, 17, 261–265. https://doi.org/10.1016/j.jval.2013.12.002CrossRefPubMedPubMedCentral

29.

Suñer, I. J., Kokame, G. T., Yu, E., Ward, J., Dolan, C., & Bressler, N. M. (2009). Responsiveness of NEI VFQ-25 to changes in visual acuity in neovascular AMD: Validation studies from two phase 3 clinical trials. Investigative Ophthalmology and Visual Science, 50(8), 3629–3635. https://doi.org/10.1167/iovs.08-3225CrossRefPubMed

30.

Coon, C. D., & Cook, K. F. (2018). Moving from significance to real-world meaning: Methods for interpreting change in clinical outcome assessment scores. Quality of Life Research, 27(1), 33–40. https://doi.org/10.1007/s11136-017-1616-3CrossRefPubMed

31.

Khanna, D., Krishnan, E., Dewitt, E. M., Khanna, P. P., Spiegel, B., & Hays, R. D. (2011). The future of measuring patient-reported outcomes in rheumatology: Patient-reported outcomes measurement information system (PROMIS). Arthritis Care Research, 63, S486–S490. https://doi.org/10.1002/acr.20581CrossRefPubMedPubMedCentral

32.

Cook, K. F., Cella, D., & Reeve, B. B. (2019). PRO-Bookmarking to estimate clinical thresholds for patient-reported symptoms and function. Medical Care, 57(Suppl 1), S13–S17. https://doi.org/10.1097/mlr.0000000000001087CrossRefPubMed

33.

Lewis, D. M., Mitzel, H. C., Green, D. R., & Patz, R. J. (1999). The Bookmark standard setting procedure. McGraw Hill.

34.

Karantonis, A., & Sireci, S. G. (2006). The bookmark standard-setting method: A literature review. Educational Measurement Issues and Practice, 25(1), 4–12. https://doi.org/10.1111/j.1745-3992.2006.00047.xCrossRef

35.

Bingham, C. O., Butanis, A. L., Orbai, A. M., Jones, M., Ruffing, V., Lyddiatt, A., Schrandt, M. S., Bykerk, V. P., Cook, K. F., & Bartlett, S. J. (2021). Patients and clinicians define symptom levels and meaningful change for PROMIS pain interference and fatigue in RA using bookmarking. Rheumatology, 60(9), 4306–4314. https://doi.org/10.1093/rheumatology/keab014CrossRefPubMedPubMedCentral

36.

Ricker, K. L. (2006). Setting cut-scores: A critical review of the Angoff and modified Angoff methods. The Alberta Journal of Educational Research, 52(1), 53–64.

37.

Schindler, N., Corcoran, J., & DaRosa, D. (2007). Description and impact of using a standard-setting method for determining pass/fail scores in a surgery clerkship. The American Journal of Surgery, 193(2), 252–257. https://doi.org/10.1016/j.amjsurg.2006.07.017CrossRefPubMed

38.

Melican, G. J., Mills, C. N., & Plake, B. S. (1989). Accuracy of item performance predictions based on the Nedelsky standard setting method. Educational and Psychological Measurement, 49(2), 467–478.CrossRef

39.

Norcini, J. J. (2003). Setting standards on educational tests. Medical Education, 37(5), 464–469. https://doi.org/10.1046/j.1365-2923.2003.01495.xCrossRefPubMed

40.

Livingston, S. A., & Zieky, M. J. (1989). A comparative study of standard-setting methods. Applied Measurement in Education, 2(2), 121–141. https://doi.org/10.1207/s15324818ame0202_3CrossRef

41.

Park, J., Ahn, D. S., Yim, M. K., & Lee, J. (2018). Comparison of standard-setting methods for the Korean radiological technologist licensing examination: Angoff, Ebel, bookmark, and Hofstee. Journal of Educational Evaluation for Health Professions, 15, 32. https://doi.org/10.3352/jeehp.2018.15.32CrossRefPubMedPubMedCentral

Titel: The challenges inherent with anchor-based approaches to the interpretation of important change in clinical outcome assessments
Auteurs: Kathleen W. Wyrwich
Geoffrey R. Norman
Publicatiedatum: 18-11-2022
Uitgeverij: Springer International Publishing
Gepubliceerd in: Quality of Life Research / Uitgave 5/2023
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI: https://doi.org/10.1007/s11136-022-03297-7

Bohn Stafleu van Loghum

Deel dit onderdeel of sectie (kopieer de link)

Abstract

Purpose

Methods

Results

Conclusions

Log in om toegang te krijgen

Andere artikelen Uitgave 5/2023

Introduction to the special section: “Methodologies and considerations for meaningful change”

Evaluating health-related quality of life and subjective wellbeing among infertility patients: a cross-sectional study in mainland China

Likely change indexes do not always index likely change; moreover, there is no need for them

Pre-treatment quality of life in patients with salivary gland cancer in comparison with those of head and neck cancer patients

A qualitative study to examine meaningful change in physical function associated with weight-loss

Using qualitative methods to establish the clinically meaningful threshold for treatment success in alopecia areata