Toward more efficient diagnostic criteria sets and rules: The use of optimization approaches in addiction science

Highlights

• Optimization approaches can be used to improve clinical assessment.

• Optimal diagnostic criteria sets can be derived using complete enumeration.

• Complete enumeration can identify criteria sets common to multiple substances.

Abstract

Psychiatric diagnostic systems, such as The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), use expert consensus to determine diagnostic criteria sets and rules (DCSRs), rather than exploiting empirical techniques to arrive at optimal solutions (OS). Our project utilizes complete enumeration (i.e., generating all possible subsets of item combinations A and B with all possible thresholds, T) to evaluate all possible DCSRs given a set of relevant diagnostic data. This method yields the entire population distribution of diagnostic classifications (i.e., diagnosis of the disorder versus no diagnosis) produced by a set of dichotomous predictors (i.e., diagnostic criteria). Once unique sets are enumerated, optimization on some predefined correlate or predictor will maximally separate diagnostic groups on one or more, disorder-specific “outcome” criteria. We used this approach to illustrate how to create a common Substance Use Disorder (SUD) DCSR that is applicable to multiple substances. We demonstrate the utility of this approach with respect to alcohol use disorder and Cannabis Use Disorder (CUD) using DSM-5 criteria as input variables. The optimal SUD solution with a moderate or above severity grading included four criteria (i.e. 1) having a strong urge or craving for the substance (CR), 2) failure to fulfill major role obligations at work school or home (FF), 3) continued use of the substance despite social or interpersonal problems caused by the substance use (SI) and 4) physically hazardous use (HU)) with a diagnostic threshold of two. The derived DCSR was validated with known correlates of SUD and performed as well as DSM-5. Our findings illustrate the value of using an empirical approach to what is typically a subjective process of choosing criteria and algorithms that is prone to bias. The optimization of diagnostic criteria can reduce criteria set sizes, resulting in decreased research, clinician, and patient burden.

Introduction

Valid and reliable diagnostic criteria sets and rules (DCSRs) are fundamental to clinical research and practice. Current diagnostic standards, such as the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM–5; American Psychiatric Association [APA], 2013) utilize expert consensus informed by relevant literature to develop DCSRs of psychopathology (Frances & Widiger, 2012; Hasin et al., 2013). For example, in revising the DSM, the DSM-5 Substance-Related Disorders Workgroup (Hasin et al., 2013) tasked itself with answering questions such as: “Which if any diagnostic criteria can be dropped?” and “What should the diagnostic threshold be?” (Hasin et al., 2013). The ability to answer questions such as these is limited by the inherent subjectivity that permeates the consensus process (Frances & Widiger, 2012; Wakefield, 2015). However, statistical techniques can be used to empirically derive new DCSRs where the process is transparent and biases are made explicit.

The optimization procedure described here (Steinley et al., 2016, Steinley et al., 2016; Stevens et al., 2018), can empirically derive DCSRs based on a priori clinical correlates and outcomes that serve as optimization criteria (OptCrit). Raffo and colleagues (2018) have successfully demonstrated that a similar method could be used to create short-forms of diagnosis by optimizing the correlations between the full DSM-5 Alcohol Use Disorder (AUD) criteria set with diagnostic thresholds (i.e., the number of criteria in a set one must endorse to diagnose) of one or two. By optimizing on the basis of part-whole correlations, Raffo et al.'s approach is likely influenced by intrinsic correlated error, a limitation not shared with our current approach. Additionally, the Raffo et al., (2018) approach employed a post hoc evaluation of a fixed number of diagnostic thresholds subsequent to short-form derivation, in effect, evaluating only a restricted subset of possible criteria sets and diagnostic rules and thus not guaranteed to identify an optimal solution (OS). The current procedure, however, is optimizing on external criteria to derive novel diagnoses rather than creating a shortened form of the diagnostic sets. This approach uses complete enumeration to compare all possible criteria set sizes and diagnostic thresholds from a selected set of criteria. Table 1 illustrates complete enumeration on an 11 item criteria set varying the diagnostic threshold from 1 to 11, producing 11,264 DCSRs. Once completely enumerated, the distribution of all measures of separation produced can be used to identify where the diagnostic grouping (i.e., those diagnosing under the given rule versus those not diagnosed by the rule) are meaningfully distinguishable using some objective function, such as maximizing the effect size. Here, we extend our previous work (Steinley et al., 2016, Steinley et al., 2016; Stevens et al., 2018) by deriving a common DCSR across alcohol and cannabis in order to illustrate how this approach can be used to identify a single OS common to alcohol and other drugs of abuse (e.g., Budney, 2006). Although the focus of the current paper is to describe how optimal DCSRs can be derived, applications using the current approach can be found in optimizing the diagnosis of Alcohol Use Disorder (AUD; Boness et al., 2018), resulting in variable, equally performing diagnostic rules (i.e., set size of 9 with a diagnostic threshold of three or a set size of five with a diagnostic threshold of 2).