The validity of using patient self-report to assess psychotic symptoms in schizophrenia☆
Introduction
In schizophrenia, reliable and valid measures of psychosis are needed for clinical care, research and quality improvement. In clinical care, ratings of psychosis severity over time are useful when determining the best antipsychotic medication and dosage for individual patients. In research, there are efforts to conduct more mental health research in typical treatment settings to increase external validity and generalizability of results. In quality improvement, it is helpful to know which patients have high levels of psychosis so that their care can be improved. For each of these purposes, psychosis measurement can be performed by self-report or by clinician-rated assessments. It has proven quite difficult to implement routine, clinician ratings as clinicians are generally not trained in these instruments and resist performing these assessments. Further, using trained staff to perform these assessments is often not affordable at typical clinics. Self-report of psychosis is potentially easier to implement in routine clinical settings, and high levels of reliability are possible. In some psychopathology domains, such as depression, self-report and clinician-rated assessments are highly correlated. However, the extent to which this is true in the measurement of psychosis is poorly understood. This study examined the validity of using self-report to assess psychosis in a random sample of patients from a large, busy mental health clinic.
Routine assessment of psychotic symptoms in clinical settings using valid and reliable instruments is of value as it can serve as a psychiatric “vital sign” allowing doctors to adjust treatment as needed. For example, routine assessment can aid in evaluating individuals for psychotic exacerbation, examining medication treatment response over time to inform changes in dosage or medications, and identifying potential candidates for treatment refractory interventions such as clozapine treatment. At present, psychosis assessment is not routinely performed or documented in medical records (Cradock et al., 2001). In addition to being a potential clinical problem, this has also made it difficult to engage in quality improvement efforts that identify unmet service needs so that evidence-based interventions can be delivered (Young et al., 2004b).
Standardized and operationally defined clinician-administered instruments are considered to be the gold standard for measurement of psychiatric symptoms including psychosis. The UCLA version of the Brief Psychiatric Rating Scale (BPRS; Lukoff et al., 1986) is one such clinician-administered measure which evaluates a broad range of psychiatric symptoms and is among the most often used measures of symptom outcome in psychiatric populations (Burlingame et al., 2005). The BPRS is available in the public domain, is sensitive to change in symptom severity, and its extensive use has produced norms for a variety of populations. The primary challenge in using the BPRS, or similar instruments such as the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987), is that intensive training is required to achieve adequate reliability, and ongoing participation in a quality assurance program is required to prevent rater drift (Ventura et al., 1993). Further, the BPRS often takes 15–20 min or more to administer and rate. The lengthy training and administration time of the BPRS greatly limits its use in clinical settings despite its potential usefulness.
The limitations of clinician-based assessments have also presented a challenge to researchers. To increase external validity and the generalizability of results, there has been a national drive to increasingly conduct mental health research in typical treatment settings. An obstacle to such research has been that treatment providers are not reliable on standardized instruments such as the BPRS, and training is often not feasible. Further, treatment providers are often resistant to doing routine assessments with structured instruments. For example, the VA has mandated that clinicians routinely complete GAF scores on their patients, but these scores are often not done, and when they are, the results are generally not reliable or valid (Niv et al., in press). These limitations of clinician-based assessments underscore the need for identifying alternative methods of symptom measurement.
Self-report measures have been utilized less often than interviewer-based measures in research examining symptom outcomes (Burlingame et al., 2005). However, advantages of self-report measures are that they require little training, require less staff time and can be easily scored and interpreted. The need for brief, reliable, and valid screening instruments of psychosis led us to search for measures that may be used in situations where use of the BPRS is not feasible. One leading measure which meets this need is the Revised Behavior and Symptom Identification Scale (BASIS-R; Eisen et al., 2004), a 24-item self-report instrument that measures six domains: psychosis, depression/functioning, interpersonal problems, alcohol/drug use, self-harm, and emotional lability. The BASIS-R is very widely used, and recent research supports its reliability (Eisen et al., 2004).
The present study utilizes data from a recent VA research project, Enhancing QUality-of-care In Psychosis (EQUIP; Young et al., 2004a). We evaluate the relationship between BPRS assessments performed by interviewers trained to high levels of reliability and BASIS-R ratings completed by patients at the same point in time. We examine the extent to which psychosis severity measured using the BASIS-R is associated with psychosis severity using the BPRS in individuals with schizophrenia, and evaluate the sensitivity and specificity of the BASIS-R in screening for significant psychosis using the BPRS as the gold standard.
Section snippets
Materials and methods
This study was conducted in 2003 at the mental health clinic of the West Los Angeles Veterans Healthcare Center. Patients were eligible if they were 1) at least 18 years old, 2) had a diagnosis of schizophrenia or schizoaffective disorder as determined by an abbreviated version of the Structured Clinical Interview for the DSM-IV, Patient Edition, Version 2.0 (First et al., 1995, Young et al., 1998), and 3) had at least one visit with a psychiatrist during a 4-month sampling period. We used
Correlations between BASIS-R psychosis scores and BPRS psychosis items
Pearson correlations between the weighted BASIS-R psychosis score and BPRS psychosis items were all in the moderate range (suspiciousness: r = .53; hallucinations: r = .59; unusual thought content: r = .49; p < .001 for all). Similar correlations were found between the unweighted BASIS-R psychosis score and BPRS items (suspiciousness: r = .46; hallucinations: r = .62; unusual thought content: r = .44; p < .001 for all).
ROC analyses
Based on the BPRS psychosis classifications, 68% of the sample experienced moderate or
Discussion
Both the weighted and unweighted BASIS-R psychosis scores were moderately correlated with all three psychosis items on the BPRS suggesting that the BASIS-R can be utilized in assessing severity of psychosis in clinical and research settings. According to the BPRS ratings, 68% of patients experienced moderate or greater psychosis, and 30% experienced severe or extremely severe psychosis. This was the prevalence of psychosis among the population of patients with schizophrenia at the West Los
Acknowledgements
This work was supported by the Department of Veterans Affairs through the Health Services Research & Development Service (RCD 00-033 and CPI 99-383) and the Desert Pacific Mental Illness Research, Education and Clinical Centers (MIRECCs); and by the NIMH UCLA-RAND Center for Research on Quality in Managed Care (MH 068639). Any opinions expressed do not necessarily represent the views of the Department of Veterans Affairs.
References (15)
- et al.
Training and quality assurance with the Structural Clinical Interview for DSM-IV (SCID-I/P)
Psychiatry Res.
(1998) - et al.
A network-based system to improve care for schizophrenia: the Medical Informatics Network Tool (MINT)
J. Am. Med. Inform. Assoc.
(2004) - et al.
Selection of Outcome Assessment Instruments for inpatients with severe and persistent mental illness
Psychiatr. Serv.
(2005) - et al.
Computer-assisted self-assessment in persons with severe mental illness
J. Clin. Psychiatry
(2004) - et al.
The accuracy of medical record documentation in schizophrenia
J. Behav. Health Serv. Res.
(2001) - et al.
The revised Behavior and Symptom Identification Scale (BASIS-R-R): reliability and validity
Med. Care
(2004) - et al.
Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders
(1995)
Cited by (0)
- ☆
All authors contributed to and have approved the final manuscript. Noosha Niv performed data analysis and led the writing of the manuscript. Amy N. Cohen collaborated on study design, implementation and data collection. Jim Mintz collaborated on study design, and statistical analyses. Joseph Ventura oversaw the training and quality assurance of the interviewers and collaborated on the manuscript. Alexander S. Young led the study design, and oversaw all aspects of data collection and analysis.