Optimizing the assessment of suicidal behavior: The application of curtailment techniques

Highlights

• This is the first study to apply curtailment to the Beck Scale for Suicide Ideation.

• A cut-off value of≥6 provided the highest level of classification accuracy for future suicidal behavior.

• Both deterministic and stochastic curtailment were able to reduce the average length.

• Stochastic curtailment resulted in the greatest reduction of items administered.

• New studies using other datasets should re-validate the optimal cut-off for risk.

Abstract

Background

Given their length, commonly used scales to assess suicide risk, such as the Beck Scale for Suicide Ideation (SSI) are of limited use as screening tools. In the current study we tested whether deterministic and stochastic curtailment can be applied to shorten the 19-item SSI, without compromising its accuracy.

Methods

Data from 366 patients, who were seen by a liaison psychiatry service in a general hospital in Scotland after a suicide attempt, were used. Within 24 h of admission, the SSI was administered; 15 months later, it was determined whether a patient was re-admitted to a hospital as the result of another suicide attempt. We fitted a Receiver Operating Characteristic curve to derive the best cut-off value of the SSI for predicting future suicidal behavior. Using this cut-off, both deterministic and stochastic curtailment were simulated on the item score patterns of the SSI.

Results

A cut-off value of SSI≥6 provided the best classification accuracy for future suicidal behavior. Using this cut-off, we found that both deterministic and stochastic curtailment reduce the length of the SSI, without reducing the accuracy of the final classification decision. With stochastic curtailment, on average, less than 8 items are needed to assess whether administration of the full-length test will result in an SSI score below or above the cut-off value of 6.

Limitations

New studies using other datasets should re-validate the optimal cut-off for risk of repeated suicidal behavior after being treated in a hospital following an attempt.

Conclusions

Curtailment can be used to simplify the assessment of suicidal behavior, and should be considered as an alternative to the full scale.

Introduction

Suicidal behavior is a major public health problem, accounting for 804,000 deaths per year (World Health Organisation, 2014). Clinical guidelines on suicidal behavior highlight the importance of assessing the risk of suicidal behavior (Jacobs and Brewer, 2004, van Hemert et al., 2012, Wasserman et al., 2012). This also applies for patients treated at emergency departments after a suicide attempt. Although an earlier suicide attempt has been shown to be the best predictor of future suicidal behavior (Hawton and van Heeringen, 2009), the assessment of suicidal ideation after an attempt may help clinicians to better differentiate between patients with acute risk and a relatively low risk for future suicidal behavior. Given the stress, the great time pressure and the need for somatic treatment of patients at emergency departments, it can be difficult to assess suicidal behavior in these settings (Verwey et al., 2007). In the Netherlands it was found that of the 14,000 patients who presented at an emergency department after a suicide attempt, only 25% were seen by a hospital psychiatrist (Kerkhof et al., 2007). Also, more than half of the patients who were treated for self-harm in English hospitals left the hospital without any form of risk assessment (Friedman et al., 2006, Kapur et al., 2004).

A simple and efficient scale for risk assessment may help to improve the assessment of suicide risk in emergency departments. Using a short screener scale, patients with an elevated risk, for whom further, more thorough assessment is required, can be identified. Given their length, commonly used scales for suicide-risk assessment, such as the Beck Scale for Suicide Ideation (Beck et al., 1979) are of limited use as screening tools (De Beurs et al., 2014, Reeve et al., 2007, Smits et al., 2011, Spijker et al., 2014). However, with the application of modern psychometric techniques such as computerized adaptive testing (CAT), it is possible to reduce the number of items to be administered, without reducing predictive accuracy (Reeve et al., 2007). Specifically, in a recent clinical study (De Beurs et al., 2014, 2015a, 2015b), it was shown that on average, four items from the SSI, instead of the full set of 19 items, were sufficient to classify patients as having an elevated risk for future suicidal behavior or not (i.e., with a cut-off value of SSI>2). Although these findings for CAT are promising, it requires dedicated software that may not be readily available in clinical settings.

In the current paper, we use a technique to shorten tests that does not require a computer or dedicated software for its application, called curtailment (Finkelman et al., 2012, Fokkema et al., 2014). The rationale behind curtailment is somewhat more intuitive than that of CAT, because it depends on observed item scores only, and does not assume a latent variable underlying observed item scores (van der Linden and Hambleton, 1997).

Curtailment always needs a pre-established cut-off value for a scale. A cut-off value for a scale determines which score on that scale best classifies patients as at risk or not at risk. When the best possible cut-off value is established, curtailment can then be used to minimize the number of items that need to be administered, to decide whether a participant would score above or below the cut-off on the full scale, or in other words, should be classified as at risk or not at risk. Simply put, with curtailment, test administration is halted when responses to the remaining items can no longer change the final classification decision (at risk, or not at risk). By allowing for early stopping of item administration, curtailment shortens questionnaire administration. With deterministic curtailment (DC), item administration is stopped as soon as the responses to the remaining items cannot change the final classification decision. It is also possible to take a non-deterministic approach to curtailment, by deriving probabilities for each of the two classification outcomes, and stopping item administration as soon as the probability of one of the classification outcomes exceeds an a-priori selected threshold value. This is called stochastic curtailment (SC; Finkelman et al., 2011). Like DC, SC requires a cut-off value. In addition, for the stochastic part, a value for γ (gamma) needs to be selected by the user. γ is the threshold for the probability that classification under SC matches that of administration of the full-length test. Although SC requires some computing, it produces simple look-up tables, with stopping criteria for every item. So, in contrast to CAT, no software is needed for administration of the questionnaire, making the results much easier to implement in daily practice. Also, the order in which items are administered remains the same as in the original scale. As a result, any unforeseen effect of the order of items can be ruled out (Fokkema et al., 2014).

In the current study, we applied deterministic and stochastic curtailment to shorten one of the most commonly used questionnaires for suicide risk assessment, the Beck Scale for Suicide Ideation (SSI; Beck et al., 1979).

Data were collected for patients who were seen by a liaison psychiatry service in a general hospital in Scotland after a suicide attempt (O’Connor et al., 2015). To our knowledge, the only published cut-off score for the SSI comes from a 20-year prospective study among 6891 psychiatric outpatients (Brown et al., 2000). Outpatients with a baseline score of SSI≥3 were seven times more likely to die by suicide than outpatients who scored less than 3 at baseline. However, this cut-off may be less appropriate in a population of patients treated for suicide attempts, as the baseline suicide ideation among those patients is likely to be higher than among psychiatric outpatients (Brown et al., 2000). Therefore, the first step in our study was to determine the best cut-off value in our sample. Next, we used the item responses and the selected cut-off value to assess the extent to which DC and SC allow for shortening of the SSI without reducing classification accuracy.