Evaluating the clinical significance of responses by psychiatric inpatients to the mental health subscales of the SF-36

Abstract

Background

The Mental Health subscales of the Medical Outcomes Short Form Questionnaire (SF-36; [Ware, J.E., Snow, K.K., Kosinski, M., Gandek, B., 1993. SF-36 Health Survey: Manual and Interpretation Guide. Boston: The Health Institute, New England Medical Center]) are increasingly being used to evaluate treatment outcomes, but data to assess the clinical significance of changes are absent. The present study applied Jacobson and Truax's [Jacobson, N.S., Truax, P. 1991. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology 59, 12–19] criteria for clinical significance to the mental health items of the SF-36.

Method

Admission and discharge data were collated from 1830 consecutive inpatients at a psychiatric hospital, using the SF-36, the Depression Anxiety Stress Scale, the Quality of Life Enjoyment and Satisfaction Questionnaire and the clinician-rated Health of the Nation Outcome Scale.

Results

Appropriate improvement cut-off scores for the mental health subscales of the SF-36 are reported, and significant differences were found between outcome groups according to clinically significant improvement.

Limitations

Clinical significance as a means of assessing outcome should be used with caution in inpatient settings, as further improvement is often expected upon discharge from the hospital.

Conclusions

Assessing clinically significant improvement is an effective means of measuring treatment outcome in terms of quality of life and symptom improvement in psychiatric care.

Introduction

Assessment of outcomes is integral for guiding and refining professional practice. It is difficult however to move from randomized controlled trials reporting statistically significant average results to evaluating routine clinical practice, where the focus is on clinically significant change among individuals. Statistical significance, which provides information on whether improvement can be considered to be reliable rather than a consequence of chance variability, is often taken to be indicative of well-being. However, the mean response of a sample may improve as a result of a target intervention, yet the response may not be clinically meaningful and the effect size small. These limitations are problematic for clinical practice in that treatment efficacy studies provide an illustration of group response but fail to address the degree of improvement experienced by individual patients. Likewise, the goal of clinical practice is not only to bring about reliable change, but to ensure that the change involves meaningful improvements in presenting problems.

The two-part criteria for clinical significance (Jacobson and Truax, 1991) provide a more stringent design for outcome assessment and address these issues. To determine whether clinically significant change has been achieved, the treatment response should be sufficiently large to demonstrate statistical reliability, but it should also constitute a meaningful symptom reduction. Accordingly, for a patient to have made clinically meaningful change (recovery), two criteria must be met: statistically reliable change (as measured by the reliable change index) must have occurred during the therapy period, and the degree of change must have exceeded a cut-off such that a patient's scores move from being within a dysfunctional population into that of a non-patient or functional population (Jacobson and Truax, 1991). As such, the methodology defines the meaningfulness of a clinical outcome in a way that is consistent with both scientific and clinical interpretations of successful outcome.

A standardized approach to classifying patient treatment response is particularly useful for assessing an individual's progress in relation to expected outcomes. By providing a precise method for classifying patients as “changed” or “unchanged”, recovery is operationalized in a more useful manner. It can be applied to any disorder or treatment type, and thus may facilitate comparison between studies, and more importantly, patient groups. Clinical significance, unlike other methods of outcome assessment, provides information concerning the variability of response to treatment within the sample and illustrates to the clinician the proportion of all outcomes that can be deemed to be successful.

The approach outlined by Jacobson and Truax is becoming broadly applicable as psychiatric services have become increasingly involved in the routine collection of data with the intention of evaluating outcomes and improving practice. The routine collection of CORE-Outcome Measures data in the United Kingdom has been employed in conjunction with the principles of evidence-based practice (Barkham et al., 2001). This movement reflects similar processes in outcome evaluation worldwide. The Medical Outcomes Short Form Questionnaire (SF-36; Ware et al., 1993) is a patient self-report measure of health status that has been assessed across many patient groups of various diagnoses and socio-economic backgrounds (Ware, 1996) making it an attractive measure for comparative study and benchmarking. Yet, there is still little information available regarding the utilization of the SF-36 as a measure of patient outcome and thus it is important to determine appropriate benchmarks for identifying reliable and meaningful change when used in psychiatric services.

The SF-36 is freely available and easy to administer. It is a patient self-report measure that assesses eight health concepts, where high scores indicate a healthier level of functioning. In particular, four subscales consisting of 14 items are appropriate to the measure of mental health treatment outcome: Vitality, Role Function, Social Function and Mental Health. These subscales measure the distress and disability associated with psychological disorder and are not disease- or treatment-specific (Page et al., 2001). The Vitality subscale is used for assessing energy level and fatigue, and for differentiating levels of subjective well-being. Role Function assesses health-related limitations in occupational or routine activity functioning; whilst the Social Function subscale investigates the impact of psychological problems on social activities. The Mental Health subscale consists of 5 items that cover four major mental health dimensions (depression, anxiety, loss of behavioral or emotional control, and psychological well-being) (Ware et al., 1993).

Arguably the SF-36 subscales relevant to mental health are an appropriate measure for assessing treatment outcomes across patient groups and for individual patients. However, psychiatric inpatients represent a group for whom the concept of clinical significance needs to be applied with some caution. Inpatients may receive qualitatively different treatment from outpatients (e.g., ECT), the structure of any psychological therapy can be more intense or time-limited, and they may not have the opportunity to generalize therapy change processes to daily routine at home. Furthermore, in the same way that a surgical patient may be discharged once effectively treated but still requiring additional time to return to healthy function, psychiatric inpatients may be treated and then discharged with the expectation of continued improvement due to provision treatment and management in the community. In addition, inpatient groups tend to be characterized by a more severe level of disturbance than outpatient groups. Therefore, to determine the validity of outcome assessment according to clinical significance criteria, evaluation within a clinical inpatient sample is essential.

To facilitate the use of clinical significance in routine practice, the current study aims to determine the values required to calculate the twin criteria of reliable change and appropriate cut-off scores for the Vitality, Social Function, Role Function and Mental Health subscales among psychiatric inpatients. It is expected that patients who attain clinically significant change will demonstrate a meaningful reduction in symptoms and an improvement in quality of life to a degree that is consistent with the normal population. As such, the current study intends to assess the validity of clinical significance when administering the SF-36 as a measure of psychiatric outcome.