Validity Concerns With Multiplying Ordinal Items Defined by Binned Counts
An Application to a Quantity-Frequency Measure of Alcohol Use
Abstract
Social and behavioral scientists often measure constructs that are truly discrete counts by collapsing (or binning) the counts into a smaller number of ordinal responses. While prior quantitative research has identified a series of concerns with similar binning procedures, there has been a lack of study on the consequences of multiplying these ordinal items to create a desired index. This measurement strategy is incorporated in many research applications, but it is particularly salient in the study of substance use where the product of ordinal quantity (number of drinks) and frequency (number of days) items is used to create an index of total consumption. In the current study, we demonstrate both analytically and empirically that this multiplicative procedure can introduce serious threats to construct validity. These threats, in turn, directly impact the ability to accurately measure alcohol consumption.
References
2003). Assessing alcohol problems: A guide for clinicians and researchers (2nd ed.). Bethesda, MD: U.S. Dept. of Health and Human Services, Public Health Service, National Institutes of Health, National Institute on Alcohol Abuse and Alcoholism.
(1983). The cost of dichotomization. Applied Psychological Measurement, 7, 249–253. doi: 10.1177/014662168300700301
(2003). Methodological issues in measuring alcohol use. Alcohol Research and Health, 27, 18–29.
(2000). Towards agreement on ways to measure and report drinking patterns and alcohol-related problems in adult general population surveys: The Skarpö Conference overview. Journal of Substance Abuse, 12, 1–21. doi: 10.1016/S0899-3289(00)00037-7
(2003). The validity of self‐reports of alcohol consumption: State of the science and challenges for research. Addiction, 98, 1–12. doi: 10.1046/j.1359-6357.2003.00586.x
(1996). Estimating the extent and degree of gambling related problems in the Australian population: A national survey. Journal of Gambling Studies, 12, 161–178. doi: 10.1007/BF01539172
(1986). Quantity per occasion and consequences of drinking: A reconsideration and recommendation. The International Journal of the Addictions, 21, 1059–1079. doi: 10.3109/10826088609077255
(2008). Alcohol measurement methodology in epidemiology: Recent advances and opportunities. Addiction, 103, 1082–1099. doi: 10.1111/j.1360-0443.2008.02197.x
(2011). Negative binomial regression. New York, NY: Cambridge University Press.
(1997). The effect of question structure on self-reports of heavy drinking: Closed-ended versus open-ended questions. Journal of Studies on Alcohol, 58, 622–624.
(2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7, 19–. doi: 10.1037//1082-989X.7.1.19
(1994). Comparing usual quantity/frequency and graduated frequency scales to assess yearly alcohol consumption: Results from the 1990 US National Alcohol Survey. Addiction, 89, 407–412. doi: 10.1111/j.1360-0443.1994.tb00914.x
(2003). Recommended alcohol consumption questions. Retrieved from: www.niaaa.nih.gov/research/guidelines-and-resources/recommended-alcohol-questions
. (1993). Panic factor: Outcome variable in panic disorder. Journal of Psychiatric Research, 27, 369–377. doi: 10.1016/0022-3956(93)90064-9
(1999). Assessment methods for alcohol consumption, prevalence of high risk drinking and harm: A sensitivity analysis. International Journal of Epidemiology, 28, 219–224. doi: 10.1093/ije/28.2.219
(2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
(2006). Loss of power in logistic, ordinal logistic, and probit regression when an outcome variable is coarsely categorized. Educational and Psychological Measurement, 66, 228–239. doi: 10.1177/0013164405278580
(2012). National Survey on Drug Use and Health, 2010. Ann Arbor, MI: Inter-university Consortium for Political and Social Research.
(2011). Global status report on alcohol and health. Geneva, Switzerland: World Health Organization.
. (