Research reportEvaluating the clinical significance of responses by psychiatric inpatients to the mental health subscales of the SF-36
Introduction
Assessment of outcomes is integral for guiding and refining professional practice. It is difficult however to move from randomized controlled trials reporting statistically significant average results to evaluating routine clinical practice, where the focus is on clinically significant change among individuals. Statistical significance, which provides information on whether improvement can be considered to be reliable rather than a consequence of chance variability, is often taken to be indicative of well-being. However, the mean response of a sample may improve as a result of a target intervention, yet the response may not be clinically meaningful and the effect size small. These limitations are problematic for clinical practice in that treatment efficacy studies provide an illustration of group response but fail to address the degree of improvement experienced by individual patients. Likewise, the goal of clinical practice is not only to bring about reliable change, but to ensure that the change involves meaningful improvements in presenting problems.
The two-part criteria for clinical significance (Jacobson and Truax, 1991) provide a more stringent design for outcome assessment and address these issues. To determine whether clinically significant change has been achieved, the treatment response should be sufficiently large to demonstrate statistical reliability, but it should also constitute a meaningful symptom reduction. Accordingly, for a patient to have made clinically meaningful change (recovery), two criteria must be met: statistically reliable change (as measured by the reliable change index) must have occurred during the therapy period, and the degree of change must have exceeded a cut-off such that a patient's scores move from being within a dysfunctional population into that of a non-patient or functional population (Jacobson and Truax, 1991). As such, the methodology defines the meaningfulness of a clinical outcome in a way that is consistent with both scientific and clinical interpretations of successful outcome.
A standardized approach to classifying patient treatment response is particularly useful for assessing an individual's progress in relation to expected outcomes. By providing a precise method for classifying patients as “changed” or “unchanged”, recovery is operationalized in a more useful manner. It can be applied to any disorder or treatment type, and thus may facilitate comparison between studies, and more importantly, patient groups. Clinical significance, unlike other methods of outcome assessment, provides information concerning the variability of response to treatment within the sample and illustrates to the clinician the proportion of all outcomes that can be deemed to be successful.
The approach outlined by Jacobson and Truax is becoming broadly applicable as psychiatric services have become increasingly involved in the routine collection of data with the intention of evaluating outcomes and improving practice. The routine collection of CORE-Outcome Measures data in the United Kingdom has been employed in conjunction with the principles of evidence-based practice (Barkham et al., 2001). This movement reflects similar processes in outcome evaluation worldwide. The Medical Outcomes Short Form Questionnaire (SF-36; Ware et al., 1993) is a patient self-report measure of health status that has been assessed across many patient groups of various diagnoses and socio-economic backgrounds (Ware, 1996) making it an attractive measure for comparative study and benchmarking. Yet, there is still little information available regarding the utilization of the SF-36 as a measure of patient outcome and thus it is important to determine appropriate benchmarks for identifying reliable and meaningful change when used in psychiatric services.
The SF-36 is freely available and easy to administer. It is a patient self-report measure that assesses eight health concepts, where high scores indicate a healthier level of functioning. In particular, four subscales consisting of 14 items are appropriate to the measure of mental health treatment outcome: Vitality, Role Function, Social Function and Mental Health. These subscales measure the distress and disability associated with psychological disorder and are not disease- or treatment-specific (Page et al., 2001). The Vitality subscale is used for assessing energy level and fatigue, and for differentiating levels of subjective well-being. Role Function assesses health-related limitations in occupational or routine activity functioning; whilst the Social Function subscale investigates the impact of psychological problems on social activities. The Mental Health subscale consists of 5 items that cover four major mental health dimensions (depression, anxiety, loss of behavioral or emotional control, and psychological well-being) (Ware et al., 1993).
Arguably the SF-36 subscales relevant to mental health are an appropriate measure for assessing treatment outcomes across patient groups and for individual patients. However, psychiatric inpatients represent a group for whom the concept of clinical significance needs to be applied with some caution. Inpatients may receive qualitatively different treatment from outpatients (e.g., ECT), the structure of any psychological therapy can be more intense or time-limited, and they may not have the opportunity to generalize therapy change processes to daily routine at home. Furthermore, in the same way that a surgical patient may be discharged once effectively treated but still requiring additional time to return to healthy function, psychiatric inpatients may be treated and then discharged with the expectation of continued improvement due to provision treatment and management in the community. In addition, inpatient groups tend to be characterized by a more severe level of disturbance than outpatient groups. Therefore, to determine the validity of outcome assessment according to clinical significance criteria, evaluation within a clinical inpatient sample is essential.
To facilitate the use of clinical significance in routine practice, the current study aims to determine the values required to calculate the twin criteria of reliable change and appropriate cut-off scores for the Vitality, Social Function, Role Function and Mental Health subscales among psychiatric inpatients. It is expected that patients who attain clinically significant change will demonstrate a meaningful reduction in symptoms and an improvement in quality of life to a degree that is consistent with the normal population. As such, the current study intends to assess the validity of clinical significance when administering the SF-36 as a measure of psychiatric outcome.
Section snippets
Participants
The sample consisted of consecutive inpatients treated at a private psychiatric facility over a 2-year period for whom questionnaire data were available at both admission and discharge. Each patient was diagnosed by their treating psychiatrist according to ICD-10-AM criteria (National Centre for Classification in Health, 2002) and the sample comprised mostly of people with primary diagnoses of affective/mood (66%), and anxiety (22%) disorders, as well as schizophrenia (4%) and substance use
Criteria for clinical significance
Utilising data provided by the Australian Bureau of Statistics (1997) and patient data collected at Perth Clinic, cut-off scores were calculated for the Vitality, Social Function, Role Function and Mental Health subscales (see Table 1).
Calculating reliable change
Due to the high reliability coefficients for the Vitality and Mental Health subscales, it is possible to gauge an individual's response to treatment for these subscales. The reliable change index (RCI) is determined according to the individual patient's scores.
Discussion
Assessment of patient outcome data according to Jacobson and Truax's (1991) criteria for clinical significance provides a measure of improvement that is both statistically and clinically useful. Individual patients' response to treatment may now be gauged according to the twin criteria of reliable and meaningful change. The SF-36 subscales relevant to mental health are particularly valuable for comparing treatment response data as they are easy and quick to administer, reliable, valid and
Acknowledgements
The assistance and support provided by Moira Munro is greatly appreciated.
References (22)
- et al.
Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales (DASS) in clinical groups and a community sample
Psychological Assessment
(1998) - Australian Bureau of Statistics National Health Survey: SF-36 population norms, Australia, 1995. Canberra: ABS, 1997...
- et al.
Clinical significance methods: a comparison of statistical techniques
Journal of Personality Assessment
(2004) - et al.
Service profiling and outcomes benchmarking using the CORE-OM: toward practice-based evidence in the psychological therapies
Journal of Consulting and Clinical Psychology
(2001) - et al.
Validating the SF-36 health survey questionnaire: new outcome measure for primary care
British Medical Journal
(1992) - et al.
The Depression Anxiety Stress Scales (DASS): normative data and latent structure in a large non-clinical sample
British Journal of Clinical Psychology
(2003) - et al.
Quality of Life Enjoyment and Satisfaction Questionnaire: a new measure
Psychopharmacology Bulletin
(1993) - et al.
The psychotherapy dose–effect and its implications for treatment delivery services
Clinical Psychology: Science and Practice
(2002) - et al.
Clinical significance: a statistical approach to defining meaningful change in psychotherapy research
Journal of Consulting and Clinical Psychology
(1991) - et al.
Methods for defining and determining the clinical significance of treatment effects: description, application and alternatives
Journal of Consulting and Clinical Psychology
(1999)
Manual for the Depression Anxiety Stress Scales
Cited by (73)
Mental Health Recovery Using the Individual Recovery Outcomes Counter (I.ROC) in a Community Rehabilitation Team: A Service Evaluation
2023, Journal of Psychosocial Rehabilitation and Mental HealthAblative fractional carbon dioxide laser improves quality of life in patients with extensive burn scars: A nested case–control study
2022, Lasers in Surgery and MedicinePostoperative Care and Quality of Life After Flexible Ureteroscopy
2022, Flexible Ureteroscopy