Neurocognitive diagnosis and cut-off scores of the Screen for Cognitive Impairment in Psychiatry (SCIP-S)

https://doi.org/10.1016/j.schres.2009.08.005Get rights and content

Abstract

Objectives

To demonstrate the ability of the Screen for Cognitive Impairment in Psychiatry (SCIP-S) to discriminate between cognitively-impaired individuals and those with adequate functioning in a sample of schizophrenic and bipolar patients, as well as in a control group.

Methods

The SCIP-S, together with a full neuropsychological battery, was administered to three groups: patients with schizophrenia, patients diagnosed with bipolar disorder I, and controls. The battery scores were used to perform a standardization with respect to the control group and this served to determine the comparison groups (cognitively impaired versus unimpaired) for each of the subtests of the SCIP-S. A full analysis of decision validity was conducted on the basis of receiver operating characteristic curves (sensitivity and specificity, + LR and − LR, PPV and NPV).

Results

All the subtests yielded adequate values for sensitivity and specificity with the proposed cut-off points, while the total score of the SCIP (< 70) was associated with a sensitivity of 87.9 and specificity of 80.6.

Conclusions

The SCIP-S shows adequate decision validity as a screening tool for cognitive deficit in patients diagnosed with schizophrenia or bipolar disorder.

Introduction

Cognitive impairment in schizophrenia and bipolar disorder is important due to the repercussions it has on the diagnostic, therapeutic and rehabilitative process. Indeed, its presence and the degree and type of deficit have a key influence on many clinical decisions and care management plans, since such impairment determines the patient's autonomy in a number of functions and capacities, including illness awareness, therapeutic compliance, and the inability to remember medical appointments or various aspects of psychosocial functioning (Green, 1996, Tabarés-Seisdedos et al., 2008).

The importance of this aspect has given rise to a large body of research which has described different types of neuropsychological deficit and degrees of impairment. Patients with schizophrenia and bipolar disorder exhibit a wide range of cognitive deficits (Table 1), but the same underlying factor structure describes their neuropsychological functioning in both groups. However, the profile of impairment varies between schizophrenic and bipolar disorders, with the schizophrenic patients having a worse functioning (Czobor et al., 2007).

Also, the heterogeneity of the type and degree of deficit, influenced by the different patterns of cognitive impairment and the phase of the disease process (Saykin et al., 1994), makes it necessary to apply exhaustive and detailed test batteries, and to use numerous specialized tools for measuring or detecting different cognitive abilities that are impaired in some patients but not in others. Recently, the MATRICS (Measurement and Treatment Research to Improve Cognition in Schizophrenia) initiative of the National Institute of Mental Health (Green and Nuechterlein, 2004, Kern et al., 2004) has sought to unify and standardize the type of deficits to be measured and the tests to be used with the objective of developing new and effective treatments for the neurocognitive deficits suffered by schizophrenic patients.

Currently, the various cognitive functions proposed by MATRICS (Nuechterlein et al., 2004) are assessed by specialists using neuropsychological batteries that take at least 60–120 min to administer. Furthermore, the tools used are mostly derived from traditional neuropsychology and have not been specifically adapted or normed for a psychiatric population. This is problematic not only in terms of the potential difficulties with evaluating or interpreting some of the functions assessed, but also because such tests are difficult to administer in large patient or population samples, which require more cost-effective screening tools.

The tools used to support clinical decision-making must be studied as regards their decision validity and corresponding sensitivity and specificity, as well as their optimum cut-off points, all of which are key aspects when it comes to making accurate diagnoses. In this regard, an analysis based on the receiver operating characteristic curve (ROC; Metz, 1978) aims to evaluate the ability of a test to discriminate between alternative states of health or conditions of individuals (i.e., diagnostic accuracy), and in so doing it enables more accurate decisions to be made following test administration.

In recent years a number of scales designed specifically for the psychiatric population and which are quicker to administer than traditional batteries have been developed, and these have shown adequate psychometric properties of reliability and validity. Examples include Cognistat (Kiernan et al., 1987), the Brief Cognitive Assessment (BCA; Velligan et al., 2004), the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph et al., 1998), the Brief Assessment of Cognition in Schizophrenia (BACS; Keefe et al., 2004), and the Screen for Cognitive Impairment in Psychiatry (SCIP; Purdon, 2005). Although the psychometric properties of these instruments have been extensively evaluated in various clinical samples (Eisenstein et al., 2002, Engelhart et al., 1999, Garcia et al., 2008, Guilera et al., 2009, Hill et al., 2008, Hobart et al., 1999, Keefe et al., 2008, Pino et al., 2008, Wilk et al., 2002), few studies have conducted a detailed analysis of their sensitivity and specificity. The exceptions include a ROC curve analysis of Cognistat in brain-damaged patients (Nøkleby et al., 2008) and of the RBANS in patients with Alzheimer's disease (Duff et al., 2008), but diagnostic validity has yet to be explored in psychiatric patients or for the other tests mentioned above. The SCIP is a simple and easy-to-administer instrument designed with the intention to assess cognitive impairment in psychiatric patients. The subtests within the SCIP quantify immediate and delayed verbal list learning, working memory, verbal fluency and psychomotor speed, all of which may be impaired in schizophrenia or bipolar disorders. The SCIP has been shown to be valid and reliable in both its English and Spanish versions (Guilera et al., 2009, Pino et al., 2006, Pino et al., 2008), but its decision validity has not yet been analyzed. The aim of the present study is to explore the ability of the SCIP to distinguish between individuals with and without cognitive impairment.

Section snippets

Samples

There were a total of 277 participants consisting of 123 patients with a schizophrenia spectrum disorder (108 schizophrenia, 13 schizoaffective disorders, and 2 schizophreniform disorders), 75 with bipolar disorder I, and 79 healthy controls statistically matched to both patients' samples by sex, age, and educational level. The average duration of illness in the 123 schizophrenic patients was 145.3 (SD = 95.1) months, and the average number of prior hospital admissions was 2.6 (SD = 3.7). Most of

Results

The mean SCIP subtest and total scores for each group of subjects were calculated after binary stratification based on the impairment demonstrated on traditional neuropsychological instruments within each cognitive domain (see Table 3). Participants with scores less than 1 standard deviation below the normal mean were assigned to a cognitive affected (A) group, and the remaining participants were assigned to a cognitive non-affected (NA). On all the subtests the group with cognitive impairment

Discussion

The aim of administering the SCIP is to provide an initial objective approximation of an individual's cognitive ability and, in the event that certain deficits or diagnostic queries are detected, to pave the way for a more detailed assessment of the person's cognitive functioning. It should be remembered that screening tests must show high sensitivity and a high NPV in order to minimize the rate of false negatives, even if this leads to a certain increase in the number of false positives, in

Role of funding source

This study was financed by Pfizer Spain and supported by projects 2007FIC00736 and 2005SGR00365 of the “Departament d’Universitats, Recerca i Societat de la Informació de la Generalitat de Catalunya”, and SEJ2005-09144-C02-02/PSIC of the “Ministerio de Educación y Ciencia de España”. This study was also supported by a grant from the Spanish Ministry of Health, Instituto de Salud Carlos III, RETICS RD06/0011 (REM-TAP Network). Pfizer Spain participated in the design of the study and engaged a

Contributors

Emilio Rojo, Oscar Pino, Georgina Guilera and Juana Gómez were responsible for analysis of data, interpretation of data and writing of manuscript. Eduard Vieta, Rafael Tabarés-Seisdedos, Nuria Segarra, Anabel Martínez-Arán, Manuel Franco, Manuel J. Cuesta, Benedicto Crespo-Facorro, Miguel Bernardo, and Scot E Purdon were responsible for interpretation of data and writing of manuscript. All authors approved the final manuscript. Francisco Mesa and Javier Rejas participated in the design of the

Conflict of interest

Javier Rejas and Francisco Mesa are employed by Pfizer Spain. All other authors declare that they have no conflicts of interest.

Acknowledgements

Authors wish to thank Spanish Working Group in Cognitive Function (see Appendix) and Silvia Martínez (European Biometric Institute, Barcelona, Spain) for their support and help supporting the performing of this project.

References (49)

  • Tabarés-SeisdedosR. et al.

    Neurocognitive and clinical predictors of functional outcome in patients with schizophrenia and bipolar I disorder at one-year follow-up

    J. Affect. Disord.

    (2008)
  • VelliganD.I. et al.

    A brief cognitive assessment for use with schizophrenia patients in community clinics

    Schizophr. Res.

    (2004)
  • American Psychiatric Association
  • AndreasenN.C. et al.

    The Comprehensive Assessment of Symptoms and History (CASH). An instrument for assessing diagnosis and psychopathology

    Arch. Gen. Psychiatry

    (1992)
  • Army Individual Test Battery

    Manual of directions and scoring

    (1944)
  • CzoborP. et al.

    Neuropsychological symptom dimensions in bipolar disorder and schizophrenia

    Bipolar Disord.

    (2007)
  • DickinsonD. et al.

    Social/communication skills, cognition, and vocational functioning in schizophrenia

    Schizophr. Bull.

    (2007)
  • EisensteinN. et al.

    Normative data for healthy elderly persons with the neurobehavioral cognitive status exam (Cognistat)

    Appl. Neuropsychol.

    (2002)
  • EngelhartC. et al.

    Factor structure of the Neurobehavioral Cognitive Status Exam (COGNISTAT) in healthy, and psychiatrically and neurologically impaired, elderly adults

    Clin. Neuropsychol.

    (1999)
  • EstesW.K.

    Learning theory and intelligence

    Am. Psychol.

    (1974)
  • GreenM.F.

    What are the functional consequences of neurocognitive deficits in schizophrenia?

    Am. J. Psychiatry

    (1996)
  • GuileraG. et al.

    Clinical usefulness of the Screen for Cognitive Impairment in Psychiatry (SCIP-S) scale in patients with type I bipolar disorder

    Health Qual. Life Outcomes

    (2009)
  • HamiltonM.

    A rating scale for depression

    J. Neurol. Neurosurg. Psychiatry

    (1960)
  • HarveyP.D. et al.
  • Cited by (57)

    • Screening for cognitive impairment in schizophrenia: Psychometric properties of the German version of the Screen for Cognitive Impairment in Psychiatry (SCIP-G)

      2021, Schizophrenia Research: Cognition
      Citation Excerpt :

      The SCIP in its Spanish version also showed a differentiation between bipolar I patient and healthy controls. In patients with schizophrenia spectrum disorder, bipolar disorder and healthy controls, Rojo et al. (2010) calculated a sensitivity of 0.88 and a specificity of 0.89. These results indicate that the SCIP is able to differentiate between patients and individuals with specific impairments and those without cognitive impairment.

    View all citing articles on Scopus
    View full text