Combining psychometric and biometric measures of substance use

Abstract

This paper examines the need, feasibility, and validity of combining two biometric (urine and saliva) and three self-report (recency, peak quantity, and frequency) measures of substance use for marijuana, cocaine, opioids, and other substances (including alcohol and other drugs). Using data from 337 adults with substance dependence, we used structural equation modeling to demonstrate that these multiple measures are driven by the same underlying factor (substance use) and that no single measure is without error. We then compared the individual measures and several possible combinations of them (including one based on the latent factors and another based on the Global Appraisal of Individual Needs (GAIN) Substance Frequency Scale) to examine how well each predicted a wide range of substance-related problems. The measure with the highest construct validity in these analyses varied by drug and problem. Despite their advantages for detection, biometric measures were frequently less sensitive to the severity of other problems. Composite measures based on the substance-specific latent factors performed better than simple combinations of the biometric and psychometric measures. The Substance Frequency Scale from the GAIN performed as well as or better than all measures across problem areas, including the latent factor for any use. While the research was limited in some ways, it has important implications for the ongoing debate about the proper way to combine biometric and psychometric data.

Introduction

Although frequently used in research, the accuracy of measures of substance use collected by self-report is often questioned and perceived as biased. A major point of contention remains regarding the veracity of self-report and, to a lesser degree, peer report. Whether as a result of conscious distortion, unconscious denial, or measurement artifact, the literature clearly demonstrates systematic differences in self-report and biometric measures (Amsel et al., 1976, Buchan et al., 2002, Cook et al., 1995, Darke, 1998, Dennis et al., 2003a, Harrison, 1995, Hersh et al., 1999, Katz et al., 2005, Landry et al., 2003, McNagny and Parker, 1992, Messina et al., 2000, Mieczkowski, 1990, Nelson et al., 1998, Preston et al., 1997, Skog, 1992, Stephens, 1972, Stephens and Feucht, 1993, Weatherby et al., 1994, Wish et al., 1997). Biometric screening measures clearly identify people who denied recent use, but they also miss people who readily acknowledge use and can have limited utility for quantifying the extent of use of a long period of time—which is what is wanted in many clinical and clinical research settings.

A separate but closely related concern is how to best operationalize self-reported measures of substance use. Some of the options include days of use, amount of drugs consumed (over a period or on average), days of heavy use, times used, and peak use. Research comparing these measures has shown that they do not respond the same to demand characteristics but rather follow the logic of fuzzy number sets (Matt and Wilson, 1994, Matt et al., 2003). Lennox et al. (1996) combined quantity and frequency items into a single latent variable, using two quantity and two frequency items as measured indicators of the single heavy-drinking variables. This approach argues that the two types of measures can be considered fallible effect-indicators (Bollen and Lennox, 1991) of the same construct, that their intersection can be considered a more accurate measure of substance use, and that the latent variable can measure this alcohol use on a continuum of light to heavy drinking. Lennox et al. (1996) confirmed this structure in a sample from the National Household Survey on Drug Abuse and demonstrated that the latent variable model outperformed standard approaches to measuring alcohol use, including summed score of items and the multiplicative product of quantity and frequency in predicting work-related adverse consequences. Lennox et al. (1998) also used this approach to test the unique impact of heavy drinking and alcohol dependence on adverse alcohol-related consequences.

Compared to psychometric measures, biometric measures of substance use (e.g., from urine, saliva, blood, or hair) are often tacitly accepted as a more accurate assessment of use because they are not subject to various biases that often impact the accuracy of self-report. While they are certainly less likely to be biased due to individual demand characteristics, they too are impacted by several sources of error (Buchan et al., 2002, Cone and Weddington, 1989, Del Boca and Noll, 2000, Del Boca and Darkes, 2003, Dennis et al., 2003a, Dennis et al., 2003b, Dennis et al., 2004, Feucht et al., 1994, Mieczkowski et al., 1991, Visher and McFadden, 1991, Bennett et al., 2003). The greatest threat lies in a systematic error that comes from using a static snapshot of the biological state to infer a more general consumption level over a period of time (Goldstein and Brown, 2003). For example, blood tests may only reliably detect drugs that were used in a 1-h period prior to the test, saliva tests for substances used during the prior 1–2 days, and urine tests for use during the prior 1–7 days. As such, these tests may be insensitive to longer term and more devastating chronic drug use patterns.

A second problem is that researchers frequently ignore the large individual differences in the rates at which various drugs are metabolized. Published cut points are typically based on the 50th or 80th percentiles for how long people will continue to be positive after a given use. In some cases the actual distribution can go out days or weeks further (Buchan et al., 2002). Thus, there is no perfect way of comparing these measures of “metabolites” with self-reported measures of recency, frequency, or quantity of use. Other sources of error include relying on less expensive and/or faster screening tests that do not always agree with gas chromatography/mass spectrometry (or GC/MS, the gold standard) and often suffering from processing delays (particularly with unfrozen samples), handling errors, and participants tampering with the sample.

Rather than accepting one approach or the other, most researchers favor combining the results of multiple methods (Campbell and Fiske, 1959, Cone and Weddington, 1989, Hasin et al., 2003, Lennox and Dennis, 1994, Carroll, 1995, Kranzler et al., 1997). However, there is little research comparing the construct validity of different approaches for combining self-reported and biometric measures of substance use. Some alternatives include: (a) requiring each test to indicate substance use, (b) allowing any indication of substance use to be sufficient, and (c) combining the data into a dimensional measure (via structural equation modeling or a scale) that increases as there are more (and stronger) indications of use. Since no single approach can be considered a “criterion,” it is necessary to focus on the construct validity of the alternative approaches (Cronbach and Meehl, 1955). The construct validity of a measure is established by demonstrating that the measure is related to other measures in ways that are consistent with theory. No single test or relationship is sufficient to establish the validity of a measure, but rather a pattern of relationships is required to eliminate alternative interpretations to the measure.

A tangential issue that has been recently debated in this journal is whether information should be combined by substance or across multiple substances, especially when using the measure for prognosis and to track outcomes (see Rounsaville et al., 2003, O’Brien and Lynch, 2003, Strain, 2003, Conway et al., 2003). This is an important issue because over one third of the people with abuse or dependence in the community and two thirds of those in treatment have multiple substance use disorders (SAMH, 2003a, SAMH, 2003b). While it may be useful to identify individual patterns of use (e.g., offering methadone to an opioid user), it is likely that symptoms of withdrawal, abuse/dependence, and other substance-related problems are multiply determined by all of the substances being used. Thus, in addition to looking at how to combine psychometric and biometric information within single substances, it is also important to consider what happens when this information is combined across substances. In this article we will use data from a large cohort of chronic substance users to empirically examine the relationship of multiple psychometric and biometric measures of substance use and the construct validity of different approaches for combining them.