Clinical Perspective

Response shift and outcome assessment in orthopedic surgery: is there a difference between complete and partial treatment?

Despite the success of total hip arthroplasty (THA), approximately 10%-15% of patients will be dissatisfied with their outcome. Identifying patients at risk of not achieving meaningful gains postoperatively is critical to pre-surgical counseling and clinical decision support. Machine learning has shown promise in creating predictive models. This study used a machine-learning model to identify patient-specific variables that predict the postoperative functional outcome in THA.

A prospective longitudinal cohort of 160 consecutive patients undergoing total hip replacement for the treatment of degenerative arthritis completed self-reported measures preoperatively and at 3 months postoperatively. Using four types of independent variables (patient demographics, patient-reported health, cognitive appraisal processes and surgical approach), a machine-learning model utilizing Least Absolute Shrinkage Selection Operator (LASSO) was constructed to predict postoperative Hip Disability and Osteoarthritis Outcome Score (HOOS) at 3 months.

The most predictive independent variables of postoperative HOOS were cognitive appraisal processes. Variables that predicted a worse HOOS consisted of frequent thoughts of work (β = −0.34), frequent comparison to healthier peers (β = −0.26), increased body mass index (β = −0.17), increased medical comorbidities (β = −0.19), and the anterior surgical approach (β = −0.15). Variables that predicted a better HOOS consisted of employment at the time of surgery (β = 0.17), and thoughts related to family interaction (β = 0.12), trying not to complain (β = 0.13), and helping others (β = 0.22).

This clinical prediction model in THA revealed that the factors most predictive of outcome were cognitive appraisal processes, demonstrating their importance to outcome-based research.

Prognostic Level 1.

Surgery for degenerative cervical myelopathy has shown not only to halt neurologic deterioration, but also to improve functional impairments. Despite these improvements, some patients may be dissatisfied with their outcomes. This study aims to (1) investigate discrepancies between postoperative clinical measures and self-reported health status, and (2) identify important predictors of such discrepancies.

Four hundred and seventy-nine surgical patients were prospectively enrolled in the CSM-International study at 16 global sites. At 1-year post-op, patients rated their general health status compared with their immediate preoperative status (much better, somewhat better, the same, somewhat worse, much worse). Descriptive analyses were conducted to evaluate the agreement between achieving a clinically important improvement (MCID) in function (modified Japanese Orthopedic Association [mJOA] scale) and self-reported health status. Agreement was defined as achieving the MCID on the mJOA and reporting general health as somewhat better or much better, whereas disagreement was defined as achieving MCID on the mJOA and reporting general health as the same, somewhat worse or much worse. Logistic regression analysis was used to determine factors that influence agreement between self-report of health status and functional outcomes.

A total of 395 patients had complete follow-up data at 1-year and were included in this analysis. Based on patient self-reports, 56 (14.2%) were somewhat or much worse than a year ago, 80 (20.2%) patients were the same and 259 (65.6%) patients were somewhat or much better. Thirty percent of patients who reported being somewhat or much worse were found to achieve the MCID on the mJOA; 57.5% of patients who indicated their health status were the same as before surgery also exhibited clinically meaningful improvements in functional impairment. Based on multivariate analysis, there was an increased odds of observing an agreement between self-reports of health status and functional outcomes if the patient exhibited greater improvement in mJOA upper extremity motor function at 1-year (odds ratio [OR]: 1.41, 95% confidence interval [CI] 1.03–1.93, p=.033) and reduced odds of agreement with increased age (OR for every decade: 0.66, 95% CI 0.50–0.87, p=.0035) and increased bodily pain at 1-year (OR: 0.62, 95% CI 0.49–078, p<.0001).

There was a discrepancy between changes in mJOA and self-reports of health status in patients undergoing surgery for degenerative cervical myelopathy. Increased bodily pain at 1-year, smaller improvements in postoperative upper extremity score and increased age were associated with worsened or unchanged general health status, despite clinically significant improvements in overall postoperative function.

To investigate response shift effects in spinal cord injury (SCI) over 5 years postinjury.

Prospective cohort study observed at 1, 2, and 5 years post-SCI.

Specialized SCI centers.

Sample included 1125, 760, and 219 participants at 1, 2, and 5 years post-SCI (N = 2104). The study sample was 79% men; 39% were motor/sensory complete (mean age, 44.6±18.3y).

Not applicable.

Patient-reported outcomes included the Medical Outcomes Study 36-Item Short-Form Health Survey version 2 and the Life Satisfaction-11 Questionnaire. Participant latent variable scores were adjusted for (1) potential attrition bias and (2) propensity scores reflecting risk of worse outcomes. The Oort structural equation modeling approach for detecting and accounting for response shift effects was used to test the hypothesis that people with SCI would undergo response shifts over follow-up.

The study data comprised the time after FIM scores, an objective measure of motor and cognitive function, had improved and stabilized. Three latent variables (Physical, Mental, and Symptoms) were modeled over time. The response shift model indicated uniform recalibration and reconceptualization response shift effects over time. When adjusted for these response shift effects, Physical showed small true change improvements at 2- and 5-year follow-up, despite FIM stability.

We detected recalibration and reconceptualization response shift effects in 1- to 5-year follow-up of people with SCI. Despite stable motor and cognitive function, people with SCI are adapting to their condition. This adaptation reflects a progressive disconnection between symptoms and physical or mental health, and a real improvement in the Physical latent variable.

We sought to estimate the MID on two patient-reported outcome (PRO) measures that are frequently used in multiple sclerosis (MS) clinical research: the MS Walking Scale and the MS Impact Scale-29. We anchored the Minimally Important Differences with an objective measure of ambulation, the accelerometer.

This secondary analysis used longitudinal data from an observational study of symptoms and physical activity in 269 people with Relapsing–Remitting Multiple Sclerosis. Participants completed a battery of PRO questionnaires, and then wore an accelerometer for seven days at each data collection time point every six months for 2.5 years. Statistical analysis first defined Change Groups on the basis of the performance-based accelerometer scores, anchored to 0.5 standard deviation change; then change was defined on the basis of published and linked MIDs for the PROs.

The performance-based (accelerometer) and PRO-based change distributions were stable over time. Raw scores among the accelerometer and PRO measures were associated with large effect sizes, and PRO change scores were associated with each other but not with accelerometer change scores.

These findings contradict a central assumption that may underlie clinical research studies: that a cross-sectional correlation implies that change in PROs will correspond with change in behavior/performance. Possible explanations related to accuracy of the performance-based measure, as well as response shift effects on the PROs are discussed.

Schwartz CE. Applications of response shift theory and methods to participation measurement: a brief history of a young field.

Measurement of participation in people with disability can pose psychometric and conceptual challenges. Ambiguous or paradoxical findings can occur because of differences among people or changes within people regarding internal standards, values, or conceptualization of participation. These response shifts can affect standard psychometric indices, such as reliability and validity. We focus herein on the interpretation of patient-reported outcomes and, in particular, on the cognitive appraisal processes known as response shift. We present theoretical and conceptual distinctions building on response shift theory and other current developments in health-related quality of life research to inform participation measurement research. We discuss how response shifts can influence the interpretation of reliability, validity, and responsiveness of participation measures. We then discuss the evidence for the clinical significance of response shift phenomena and describe current design, statistical, and individualized approaches for detecting response shift phenomena.

Not uncommonly, spine surgeons and physiatrists note a mismatch between patient-reported outcome measures, where one measure might indicate a good outcome and another indicates an inferior outcome after spine treatment. This may be the result of patient characteristics that lead to changes in internal standards, values, and conceptualization of their own health-related quality of life. This can result in a “moving goal post” when a self-report outcome measure is used for prepost comparisons. These “response shifts” may obfuscate relevant changes of interest to clinicians and are meaningful and worthy of study in and of themselves.

To provide a background on response shift with an emphasis on distinctions relevant to spinal interventions, both surgical and nonsurgical. To describe current methods for detecting and investigating response shift phenomena, and to propose specific hypotheses that can be tested in collaborative research.

Two types of methods will be briefly described: methods that require new data collection; and methods that use recent statistical and technical advances to implement secondary analysis of existing data. Two specific testable hypotheses for spinal disorders are suggested along with suggested methods for testing these hypotheses.

A response shift will cause the patient to use the same functional outcome report measure differently pre- and posttreatment. Response shift phenomena are likely to affect the measurement properties of standard spine outcome measures and to obfuscate differences between treatments in clinical trials and cost-effectiveness studies. They point to a need for developing strategies in clinical practice to manage response shifts so that they enhance patient well-being.