Mail surveys resulted in more reports of substance use than telephone surveys

doi:10.1016/j.jclinepi.2004.10.007

Journal of Clinical Epidemiology

Volume 58, Issue 4, April 2005, Pages 421-424

https://doi.org/10.1016/j.jclinepi.2004.10.007 Get rights and content

Abstract

Objective

To determine to what extent the substance-use information obtained in surveys is affected by method of data collection.

Study Design and Setting

Questions on the use of alcohol and drugs were administered to samples of Minnesota adults assigned to one of two conditions to test the effect of mode of administration (mail and telephone); 816 persons completed the survey, roughly one half by mail and one half by telephone.

Results

Those interviewed by telephone revealed more heavy use of alcohol, but the mail sample includes disproportionate numbers of respondents from demographic groups that exhibit less use. Controlling for these differences across modes, as well as the differential use of listed telephone numbers and addresses, reduces the effect of mode on one measure of heavy alcohol use to nonsignificance but yields significant effects of mode on others. Specifically, those in the mail condition reported higher levels of illicit drug use in the last year, alcohol use in the last month, and heavy alcohol use in the last 2 weeks.

Conclusions

The greater, and arguably more accurate, reporting of substance use, coupled with potential cost savings, suggests that researchers should consider using mail surveys for investigating substance use.

Introduction

Surveys are essential for estimating drug use and treatment need in the United States [1]. In-person interviews are believed to provide the most accurate and reliable information [2], but may be prohibitively expensive, costing approximately twice as much as analogous telephone surveys [3]. Mail surveys are far less expensive, but how they compare with telephone surveys in terms of collecting sensitive substance-use information is unclear.

Several studies have compared the mail and telephone survey responses on personal issues such as asthma [4], sexual behavior among gay males [5], prostate surgery [6], functional limitations [7], [8], urinary symptoms [9], and anxiety [10]. In general, these studies find that telephone respondents give more socially desirable responses, and mail respondents report more problems or socially proscribed behaviors. With respect to substance-use reporting, the results are less consistent. Early indications suggested an equivalence [11], but more recent studies have found that mail questionnaires elicit higher reports of use than telephone interviews [10], [12].

More research is needed to further elucidate the effect of data collection method on substance-use reports. Such investigations are especially timely because of an overall decline in survey response rates during the past few decades [13], and an even more pronounced decline in telephone response rates attributable to telemarketing, cellular telephones, and call screening devices [14]. The main objective of this research is to assess the viability of the mail survey as an alternative to the telephone interview in collecting substance-use information because of its clear cost advantage.

Section snippets

Sample

The Minnesota Adult Survey (MAS) was a statewide survey conducted between April and July 2000 and specifically designed to compare reports of substance use across mail and telephone modes. A random digit dial (RDD) sample was purchased from Survey Sampling Inc (Fairfield, CT). Listed household addresses were appended to the RDD telephone numbers if they were found in directories. Only telephone numbers with appended addresses (53.7% of the RDD sample) were eligible to be assigned to the mail

Results

Table 1 presents the sociodemographic compositions of respondents by survey mode. Gender, race, employment status, marital status, and age all varied by mode. Mail respondents were more likely to be white, ever married, and female and were on average 3.9 years older than their telephone counterparts. Telephone respondents were more likely to work full time. Comparing respondents to 2000 U.S. Census data [19] shows that the telephone sample more closely approximated population estimates, but

Discussion

Consistent with earlier research [15], [20], the response rate was lower for the mail than for the telephone survey. The sensitive nature of substance use may make respondents hesitant to return mailed questionnaires [10]. On the other hand, there is evidence that substance nonusers opt out when they learn the survey purpose, because they feel the survey does not pertain to them [21]. If substance abusers are less likely to respond to mailed surveys, the current results showing higher use in

Acknowledgments

This project was supported by contract no. 270-97-7034 under the Minnesota State Systems Development Program administered by the Division of State Programs, Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA). The contents of this report are solely the responsibility of the authors and do not necessarily represent the official views of SAMHSA, CSAT, or the Minnesota Department of Human Services. The authors express their gratitude to

References (21)

B. Galobardes et al.
Effect of the method of administration, mail or telephone, on the validity and reliability of a respiratory health questionnaire. The Spanish centers of the European Asthma Study
J Clin Epidemiol
(1998)
J.J. Perkins et al.
An examination of self- and telephone-administered modes of administration for the Australian SF-36
J Clin Epidemiol
(1998)
T. Rhodes et al.
Does the mode of questionnaire administration affect the reporting of urinary symptoms?
Urology
(1995)
J.C. Gfroerer et al.
Introduction
E.D. De Leeuw et al.
The influence of data collection method on structural models: A comparison of a mail, a telephone, and a face-to-face survey
Sociol Methods Res
(1996)
R.M. Groves et al.
Surveys by telephone: a national comparison with personal interviews
(1979)
M. Acree et al.
Mode effects in surveys of gay men: a within-individual comparison of responses by mail and by telephone
J Sex Res
(1999)
F.J. Fowler et al.
Mode effects in a survey of Medicare prostate surgery patients
Public Opin Q
(1998)
C.A. McHorney et al.
Comparisons of the costs and quality of norms for the SF-36 health survey collected by mail versus telephone interview: results from a national survey
Med Care
(1994)
L. Fournier et al.
A comparison of mail and telephone interview strategies for mental health surveys
Can J Psychiatry
(1993)

There are more references available in the full text version of this article.

Cited by (28)

Racial disparities in healthcare provider advice to quit smoking
2018, Preventive Medicine Reports
Citation Excerpt :
Household sampling (instead of a telephone survey) was used because it increases participation by racial-ethnic minority and low-income populations (Carroll et al., 2011; Cabral et al., 2003). A self-administered, anonymous, written survey (instead of an in-person interview) was used because this method decreases the socially-desirable denial of smoking and other substance use, and yields higher self-reported smoking rates (Beebe et al., 2005; Christensen et al., 2013). The study had the approval of the Institutional Review Board of East Carolina University.
African-American/Black smokers are less likely than White smokers to be told to quit smoking by healthcare providers. This preliminary study examined the predictors of being told to quit separately by race for the first time to potentially highlight the source of this racial disparity. A random, household sample of 1670 Black and White adults from a southeastern county of the United States completed a brief survey on their demographics, smoking, access to healthcare, health status, and receipt of healthcare provider advice to quit smoking. Analyses are based on the 512 Black and White smokers in that sample. The logistic regression for all smokers revealed that after controlling for demographic, healthcare, and health status variables, White smokers were 2.39 times more likely than Black smokers to have ever been told to quit smoking. The regression for Black smokers revealed that women and older people were more likely to be told to quit, and that healthcare and health status did not contribute. In the regression for White smokers, no predictor was statistically significant. These findings suggest that one possible reason that African-Americans receive cessation advice less often than Whites is that such advice varies with their age and gender, whereas for Whites this is not the case.
The effect of training on the use of tobacco-use cessation guidelines in dental settings
2012, Journal of the American Dental Association
Citation Excerpt :
We decided to assess dentists' behaviors by using a mailed patient survey within one month of the appointment to avoid depending on dental practice staff members to distribute and collect immediate patient surveys after the appointment but before leaving the dental office. Doing so would reduce the risk of patient selection bias or even falsification of data; reduce social desirability bias risk, which is greater for telephone surveys than it is for written or mailed surveys34, 35; and reduce the need to conduct expensive chart audits, which may underestimate provider performance because tobacco-use cessation counseling often is not documented.36 The generalizability of our findings is limited because we conducted our study in only three states, which are not representative of all states, and because dentist and patient participation rates were modest.
An increase in the number of dentists conducting tobacco-use cessation treatment is needed. The authors assessed the effects of high-intensity training (HIT) or low-intensity training (LIT) and reimbursement on general dentists' tobacco-use–related attitudes and treatment behaviors.
The authors randomly selected 265 dentists in three states and assigned them to one of five groups: HIT workshop groups with and without tobacco-use cessation counseling reimbursement, LIT mailed self-study groups with and without reimbursement or a control group. Outcomes at follow-up were dentists' self-reported tobacco-use–related attitudes and behaviors and patients' reports of dentists' behaviors.
Significantly more dentists in the intervention groups reported having positive attitudes and behaviors at follow-up than did dentists in the control group. Dentists in the HIT groups, however, reported assessing patients' willingness to quit and assisting them with the quitting process significantly more often than did dentists in the LIT groups. Significantly more patients of dentists in the intervention groups who used tobacco reported receiving advice and assistance from their dentists than did patients of dentists in the control group. Adding reimbursement to HIT or LIT conditions did not provide additional intervention effect.
Dentists trained by means of a workshop or self-study program used components of a recommended guideline more frequently and felt more positive toward tobacco-use cessation counseling than did dentists in the control group.
Although the workshop training was more successful than the self-study training, the latter's reach among dentists could have a more significant public health impact. The effect of reimbursement needs further study.
Survival of dental implants in post-menopausal bisphosphonate users
2010, Journal of Prosthodontic Research
To determine whether post-menopausal women with a history of bisphosphonate use are at greater risk for implant failure or osteonecrosis of the jaw (ONJ) than an age- and gender-matched population with no history of bisphosphonate use.
A retrospective chart review and phone interview was conducted of patients seen at the Mayo Clinic who had implants placed after November 2006. Bisphosphonate users were identified by medical chart review. Dental implant survival and ONJ incidence were determined in a total of 82 post-menopausal non-bisphosphonate users and 55 post-menopausal bisphosphonate users using a phone interview. Implant failures were recorded and survival percentages were calculated for comparison.
ONJ was not observed consequent to implant placement in any of the bisphosphonate users or non-users. In non-users, 163 out of 166 implants were surviving for a cumulative survival rate of 98.19%. In bisphosphonate users, 120 out of 121 implants were surviving for a cumulative survival rate of 99.17%.
Dental implants placed in post-menopausal women have the same survival potential regardless of whether patients have a history of bisphosphonate use. Bisphosphonate users who undergo dental implant surgery are at low risk for osteonecrosis of the jaw and a bisphosphonate “drug holiday” is not indicated in these patients.
Phone respondents reported less mental health problems whereas mail interviewee gave higher physical health ratings
2008, Journal of Clinical Epidemiology
Citation Excerpt :
However, the use of different data collection methods raises the question, to what extent the results of telephone and mail surveys correspond with or deviate from each other. Differences have already been demonstrated for example in surveys of patient satisfaction [7], alcohol and drug abuse [8,9] or mental health [10]. In Germany, the representative mail health surveys, which have been carried out by the Robert Koch Institute since the mid-1980s, have now been augmented by additional telephone surveys [11].
This study examines the effects of telephone and mail interview methods on the measurement of health-related quality of life.
One thousand six hundred ninety individuals aged 25–66 were interviewed randomly either by telephone or by mail. Health-related quality of life was assessed with the German SF-8.
Although respondents in the telephone survey were more willing to participate (77.4% vs. 47.5%) the difference in the accessibility of publicly available address and telephone records meant that overall more people were interviewed by mail than by telephone (53.2% vs. 46.8%). No differences occurred in terms of the sociodemographic makeup. Telephone respondents gave a more positive account of the mental dimension of their health-related quality of life; whereas mail interviews led to a better rating of the respondents' physical well-being. Gender-specific analyses indicate a slight discrepancy in the influence of the method of interviewing on men and women. Further differences were identified concerning the variance in the existence of ceiling and floor effects and the correlation between items.
Found differences are small but at least to be valued as relevant in certain settings. Therefore, we approve the use and development of factors of amendment.
Predictors of Satisfaction Among Adult Recipients of Medicaid Mental Health Services
2007, Archives of Psychiatric Nursing
This study measured service satisfaction, perceptions of service quality and general health, and overall quality of care among 787 adult recipients of Medicaid mental health services. Methods included cross-sectional retrospective design and stratified random sampling technique. Respondents were satisfied with consumer-provider relationships and were dissatisfied with functional outcomes resulting from treatment. Satisfaction was positively correlated with ratings of mental health care and the mental health component score of the SF-12. Predictors of satisfaction included ratings of mental health care and overall health. Recommendations include coordination of services that promote patient functioning and measurement of consumer satisfaction as an indicator of quality.
Psychometric assessment of tinnitus patients within clinical practice settings
2017, Tinnitus and Stress: An Interdisciplinary Companion for Healthcare Professionals

View all citing articles on Scopus

View full text

Brief ReportMail surveys resulted in more reports of substance use than telephone surveys

Abstract

Objective

Study Design and Setting

Results

Conclusions

Introduction

Section snippets

Sample

Results

Discussion

Acknowledgments

J Clin Epidemiol

J Clin Epidemiol

Urology

Introduction

The influence of data collection method on structural models: A comparison of a mail, a telephone, and a face-to-face survey

Sociol Methods Res

Surveys by telephone: a national comparison with personal interviews

Mode effects in surveys of gay men: a within-individual comparison of responses by mail and by telephone

J Sex Res

Mode effects in a survey of Medicare prostate surgery patients

Public Opin Q

Comparisons of the costs and quality of norms for the SF-36 health survey collected by mail versus telephone interview: results from a national survey

Med Care

A comparison of mail and telephone interview strategies for mental health surveys

Can J Psychiatry

Brief Report
Mail surveys resulted in more reports of substance use than telephone surveys