Elsevier

Schizophrenia Research

Volume 90, Issues 1–3, February 2007, Pages 245-250
Schizophrenia Research

The validity of using patient self-report to assess psychotic symptoms in schizophrenia

https://doi.org/10.1016/j.schres.2006.11.011Get rights and content

Abstract

Brief, reliable and valid measures of psychosis can be very useful in both clinical practice and research, and for identifying unmet treatment needs in persons with schizophrenia. This study examines the concurrent validity and receiver operating characteristics of the psychosis scale of the Revised Behavior and Symptom Identification Scale (BASIS-R). The study was conducted with 71 adults with schizophrenia who were randomly sampled from a large mental health clinic. Study participants at the West Los Angeles Veterans Healthcare Center were assessed using the BASIS-R, a subjective, self-report measure, and the UCLA Brief Psychiatric Rating Scale (BPRS), a clinician-rated measure administered by highly trained research staff. The psychosis scale of the BASIS-R shows good concurrent validity with the psychosis items on the BPRS. Using the BPRS as the gold standard for measuring psychosis, receiver operating characteristics suggest that both the weighted and unweighted versions of the BASIS-R psychosis scale adequately identify psychosis that is moderate or greater or severe. The performance of the two versions was similar. Unweighted scores are easier to calculate, and we therefore recommend cutoff scores based on the unweighted BASIS-R. We identified a cutoff score of 0.5 to best detect moderate or greater psychosis, and a cutoff score of 1.0 to best detect severe or extremely severe psychosis. The BASIS-R has potential as an assessment tool and screening instrument in schizophrenia.

Introduction

In schizophrenia, reliable and valid measures of psychosis are needed for clinical care, research and quality improvement. In clinical care, ratings of psychosis severity over time are useful when determining the best antipsychotic medication and dosage for individual patients. In research, there are efforts to conduct more mental health research in typical treatment settings to increase external validity and generalizability of results. In quality improvement, it is helpful to know which patients have high levels of psychosis so that their care can be improved. For each of these purposes, psychosis measurement can be performed by self-report or by clinician-rated assessments. It has proven quite difficult to implement routine, clinician ratings as clinicians are generally not trained in these instruments and resist performing these assessments. Further, using trained staff to perform these assessments is often not affordable at typical clinics. Self-report of psychosis is potentially easier to implement in routine clinical settings, and high levels of reliability are possible. In some psychopathology domains, such as depression, self-report and clinician-rated assessments are highly correlated. However, the extent to which this is true in the measurement of psychosis is poorly understood. This study examined the validity of using self-report to assess psychosis in a random sample of patients from a large, busy mental health clinic.

Routine assessment of psychotic symptoms in clinical settings using valid and reliable instruments is of value as it can serve as a psychiatric “vital sign” allowing doctors to adjust treatment as needed. For example, routine assessment can aid in evaluating individuals for psychotic exacerbation, examining medication treatment response over time to inform changes in dosage or medications, and identifying potential candidates for treatment refractory interventions such as clozapine treatment. At present, psychosis assessment is not routinely performed or documented in medical records (Cradock et al., 2001). In addition to being a potential clinical problem, this has also made it difficult to engage in quality improvement efforts that identify unmet service needs so that evidence-based interventions can be delivered (Young et al., 2004b).

Standardized and operationally defined clinician-administered instruments are considered to be the gold standard for measurement of psychiatric symptoms including psychosis. The UCLA version of the Brief Psychiatric Rating Scale (BPRS; Lukoff et al., 1986) is one such clinician-administered measure which evaluates a broad range of psychiatric symptoms and is among the most often used measures of symptom outcome in psychiatric populations (Burlingame et al., 2005). The BPRS is available in the public domain, is sensitive to change in symptom severity, and its extensive use has produced norms for a variety of populations. The primary challenge in using the BPRS, or similar instruments such as the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987), is that intensive training is required to achieve adequate reliability, and ongoing participation in a quality assurance program is required to prevent rater drift (Ventura et al., 1993). Further, the BPRS often takes 15–20 min or more to administer and rate. The lengthy training and administration time of the BPRS greatly limits its use in clinical settings despite its potential usefulness.

The limitations of clinician-based assessments have also presented a challenge to researchers. To increase external validity and the generalizability of results, there has been a national drive to increasingly conduct mental health research in typical treatment settings. An obstacle to such research has been that treatment providers are not reliable on standardized instruments such as the BPRS, and training is often not feasible. Further, treatment providers are often resistant to doing routine assessments with structured instruments. For example, the VA has mandated that clinicians routinely complete GAF scores on their patients, but these scores are often not done, and when they are, the results are generally not reliable or valid (Niv et al., in press). These limitations of clinician-based assessments underscore the need for identifying alternative methods of symptom measurement.

Self-report measures have been utilized less often than interviewer-based measures in research examining symptom outcomes (Burlingame et al., 2005). However, advantages of self-report measures are that they require little training, require less staff time and can be easily scored and interpreted. The need for brief, reliable, and valid screening instruments of psychosis led us to search for measures that may be used in situations where use of the BPRS is not feasible. One leading measure which meets this need is the Revised Behavior and Symptom Identification Scale (BASIS-R; Eisen et al., 2004), a 24-item self-report instrument that measures six domains: psychosis, depression/functioning, interpersonal problems, alcohol/drug use, self-harm, and emotional lability. The BASIS-R is very widely used, and recent research supports its reliability (Eisen et al., 2004).

The present study utilizes data from a recent VA research project, Enhancing QUality-of-care In Psychosis (EQUIP; Young et al., 2004a). We evaluate the relationship between BPRS assessments performed by interviewers trained to high levels of reliability and BASIS-R ratings completed by patients at the same point in time. We examine the extent to which psychosis severity measured using the BASIS-R is associated with psychosis severity using the BPRS in individuals with schizophrenia, and evaluate the sensitivity and specificity of the BASIS-R in screening for significant psychosis using the BPRS as the gold standard.

Section snippets

Materials and methods

This study was conducted in 2003 at the mental health clinic of the West Los Angeles Veterans Healthcare Center. Patients were eligible if they were 1) at least 18 years old, 2) had a diagnosis of schizophrenia or schizoaffective disorder as determined by an abbreviated version of the Structured Clinical Interview for the DSM-IV, Patient Edition, Version 2.0 (First et al., 1995, Young et al., 1998), and 3) had at least one visit with a psychiatrist during a 4-month sampling period. We used

Correlations between BASIS-R psychosis scores and BPRS psychosis items

Pearson correlations between the weighted BASIS-R psychosis score and BPRS psychosis items were all in the moderate range (suspiciousness: r = .53; hallucinations: r = .59; unusual thought content: r = .49; p < .001 for all). Similar correlations were found between the unweighted BASIS-R psychosis score and BPRS items (suspiciousness: r = .46; hallucinations: r = .62; unusual thought content: r = .44; p < .001 for all).

ROC analyses

Based on the BPRS psychosis classifications, 68% of the sample experienced moderate or

Discussion

Both the weighted and unweighted BASIS-R psychosis scores were moderately correlated with all three psychosis items on the BPRS suggesting that the BASIS-R can be utilized in assessing severity of psychosis in clinical and research settings. According to the BPRS ratings, 68% of patients experienced moderate or greater psychosis, and 30% experienced severe or extremely severe psychosis. This was the prevalence of psychosis among the population of patients with schizophrenia at the West Los

Acknowledgements

This work was supported by the Department of Veterans Affairs through the Health Services Research & Development Service (RCD 00-033 and CPI 99-383) and the Desert Pacific Mental Illness Research, Education and Clinical Centers (MIRECCs); and by the NIMH UCLA-RAND Center for Research on Quality in Managed Care (MH 068639). Any opinions expressed do not necessarily represent the views of the Department of Veterans Affairs.

References (15)

There are more references available in the full text version of this article.

Cited by (0)

All authors contributed to and have approved the final manuscript. Noosha Niv performed data analysis and led the writing of the manuscript. Amy N. Cohen collaborated on study design, implementation and data collection. Jim Mintz collaborated on study design, and statistical analyses. Joseph Ventura oversaw the training and quality assurance of the interviewers and collaborated on the manuscript. Alexander S. Young led the study design, and oversaw all aspects of data collection and analysis.

View full text