Item banks for alcohol use from the Patient-Reported Outcomes Measurement Information System (PROMIS®): Use, consequences, and expectancies

https://doi.org/10.1016/j.drugalcdep.2012.11.002Get rights and content

Abstract

Background

We report on the development and calibration of item banks for alcohol use, negative and positive consequences of alcohol use, and negative and positive expectancies regarding drinking as part of the Patient-Reported Outcomes Measurement Information System (PROMIS®).

Methods

Comprehensive literature searches yielded an initial bank of more than 5000 items from over 200 instruments. After qualitative item analysis (including focus groups and cognitive interviewing), 141 items were included in field testing. Items for alcohol use and consequences were written in a first-person, past-tense format with a 30-day time frame and 5 response options reflecting frequency. Items for expectancies were written in a third-person, present-tense format with no time frame specified and 5 response options reflecting intensity. The calibration sample included 1407 respondents, 1000 from the general population (ascertained through an internet panel) and 407 from community treatment programs participating in the National Institute on Drug Abuse (NIDA) Clinical Trials Network (CTN).

Results

Final banks of 37, 31, 20, 11, and 9 items (108 total items) were calibrated for alcohol use, negative consequences, positive consequences, negative expectancies, and positive expectancies, respectively, using item response theory (IRT). Seven-item static short forms were also developed from each item bank.

Conclusions

Test information curves showed that the PROMIS item banks provided substantial information in a broad range of severity, making them suitable for treatment, observational, and epidemiological research.

Introduction

The Patient-Reported Outcomes Measurement Information System (PROMIS®) is an NIH Roadmap initiative designed to improve self-reported outcomes using state-of-the-art psychometric methods (for detailed information, see http://www.nihpromis.org/). PROMIS has developed and calibrated item banks assessing physical, mental, and social health, consistent with the World Health Organization's tripartite framework (Cella et al., 2007). For example, there are item banks assessing physical functioning, pain, fatigue, sleep disturbance, emotional distress (depression, anxiety, and anger), and social participation, providing a comprehensive profile of health status (Buysse et al., 2010, Cella et al., 2007, Cella et al., 2010, Fries et al., 2009, Pilkonis et al., 2011, Revicki et al., 2009). PROMIS is the most ambitious attempt to date to apply models from item response theory (IRT) to health-related assessment. The PROMIS methodology involves iterative steps of comprehensive literature searches; item pooling; development of conceptual frameworks; qualitative assessment of items using expert review, focus groups, and cognitive interviewing; and quantitative evaluation of items using techniques from both classical test theory (CTT) and IRT (Cella et al., 2007, Cella et al., 2010, Hilton, 2011, Reeve et al., 2007). We report here on the development and calibration of five item banks capturing prominent aspects of alcohol use (consumption, craving, efforts at control, internal and external triggers for drinking), negative and positive consequences of alcohol use, and negative and positive expectancies regarding drinking.

There is informative previous work using IRT models for the scaling and calibration of criteria for substance abuse and dependence, including those for alcohol (Krueger et al., 2004, Langenbucher et al., 2004, Martin et al., 2006, Saha et al., 2006). These criteria are sufficiently unidimensional for calibration with IRT models, but they are high-threshold items most appropriate for clinical samples. The use of such items results in “tests” with narrow bandwidth relevant only at the severe end of the continuum of alcohol use and consequences. From a psychometric perspective, our goal was to identify items that were more normally distributed and less positively skewed in a sample that included both members of the general population who used alcohol as well as alcohol abusers. Such items will provide more information across a broader range of the continuum of alcohol use, and for this reason, they will constitute more sensitive measures of treatment outcome and result in a single metric that could be used across treatment, observational, and epidemiological settings. Thus, we were interested in alcohol use not only at the level of clinical disorders but also at lower levels of consumption, where alcohol use may still be an important health-related behavior (or risk factor) relevant to a wide range of medical and psychiatric conditions.

Section snippets

Comprehensive literature searches

The Pittsburgh PROMIS research site developed a methodology for performing comprehensive literature searches to ensure content validity and broad coverage of the alcohol domain. We performed searches in the MEDLINE, PsycINFO, and Health and Psychosocial Instruments (HaPI) databases. Details of the methodology are reported in Klem et al. (2009), and all search algorithms are available upon request. The searches generated 785 abstracts that could be linked to more than 200 unique measures of

Alcohol, drug, and tobacco use in the two samples

The most stable estimates of alcohol use came from the NIAAA questions for the past 12 months (2003), given that shorter term patterns of drinking were influenced by the recent decision to seek treatment among members of the clinical sample. Table 2 summarizes these data. Frequency of drinking did not distinguish the community and clinical samples, with “once a week” being the median for both groups over the past year. (In this context, please note that members of the clinical sample sought

Discussion

Compared with existing alcohol use instruments (which typically cover the higher end of the severity spectrum, 1–2 SDs above the mean), the PROMIS item bank for alcohol use provides more information in a range from −1 to +3 SDs among people who have had some exposure to alcohol in the past month. The item banks for negative and positive consequences and negative and positive expectancies also provide broad coverage of these constructs. The item banks for use and consequences (both positive and

Role of funding source

The Patient-Reported Outcomes Measurement Information System (PROMIS®) is a National Institutes of Health (NIH) Roadmap initiative to develop a computerized system measuring patient-reported outcomes in respondents with a wide range of chronic diseases and demographic characteristics. PROMIS was funded initially by cooperative agreements to a Statistical Coordinating Center (Evanston Northwestern Healthcare, PI: David Cella, PhD, U01AR52177) and six Primary Research Sites (Duke University, PI:

Contributors

Paul A. Pilkonis, PhD, contributed to study conception and design and took responsibility for drafting the manuscript. Lan Yu, PhD, provided data analysis and interpretation. Jason B. Colditz, BS, Nathan Dodds, BS, Kelly L. Johnston, MPH, Catherine Maihoefer, MS, LSW, Angela M. Stover, MA, Dennis C. Daley, PhD, and Dennis McCarty, PhD, contributed to study implementation (literature reviews, conceptual organization of items, item and intellectual property reviews) and data collection (focus

Conflict of interest

No conflict declared.

Acknowledgements

We acknowledge the work of our colleagues within the two research nodes (the Tri-State Appalachian node and the Western States node) affiliated with the NIDA Clinical Trials Network that collaborated in this work. Dennis Daley, PhD, is the principal investigator of the Tri-State Appalachian node, and clinical data from this node were collected in Pittsburgh, PA, where we are grateful for the help of Dorothy Sandstrom, MS, and Janis McDonald. Dennis McCarty, PhD, is the principal investigator of

References (46)

  • S. Choi

    Firestar: computerized adaptive testing simulation program for polytomous item response theory models

    Appl. Psychol. Meas.

    (2009)
  • S.W. Choi et al.

    Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms

    Qual. Life Res.

    (2010)
  • D.A. DeWalt et al.

    Evaluation of item candidates: the PROMIS qualitative item review

    Med. Care

    (2007)
  • J.F. Fries et al.

    Progress in assessing physical function in arthritis: PROMIS short forms and computerized adaptive testing

    J. Rheumatol.

    (2009)
  • T.F. Hilton

    The promise of PROMIS® for addiction

    Drug Alcohol Depend.

    (2011)
  • J.L. Horn

    A rationale and test for the number of factors in factor analysis

    Psychometrika

    (1965)
  • B.T. Jones et al.

    A review of expectancy theory and alcohol consumption

    Addiction

    (2001)
  • M.A. Kelly et al.

    Describing depression: congruence between patient experiences and clinical assessments

    Br. J. Clin. Psychol.

    (2011)
  • M.L. Klem et al.

    Building PROMIS item banks: librarians as co-investigators

    Qual. Life Res.

    (2009)
  • M.J. Kolen et al.

    Test Equating, Scaling and Linking: Methods and Practices

    (2004)
  • R.F. Krueger et al.

    Using latent trait modeling to conceptualize an alcohol problems continuum

    Psychol. Assess.

    (2004)
  • J.W. Langenbucher et al.

    An application of item response theory analysis to alcohol, cannabis, and cocaine criteria in DSM-IV

    J. Abnorm. Psychol.

    (2004)
  • R.L. Linn

    Linking results of distinct assessments

    Appl. Measure. Educ.

    (1993)
  • Cited by (54)

    • Long-Term Effects of Cognitive-Behavioral Therapy and Yoga for Worried Older Adults

      2022, American Journal of Geriatric Psychiatry
      Citation Excerpt :

      Internal consistency reliability ranged from good to excellent (α = 0.89 at baseline, α = 0.92 at Week 37). Anxiety symptoms were assessed with two measures: 1) the Patient Reported Outcomes Measurement Information System (PROMIS) Short Form v1.0 - Anxiety 8a,5,28 an eight-item measure of fear, anxious misery, and hyperarousal. Responses were converted to t-scores with a mean of 50 and standard deviation of 10 (secondary outcome).

    View all citing articles on Scopus
    View full text