Research paper
The impact of school nonresponse on substance use prevalence estimates – Germany as a case study

https://doi.org/10.1016/j.drugpo.2015.06.005Get rights and content

Highlights

  • School nonresponse in a survey on students’ substance use (ESPAD study) was simulated.

  • Systematic exclusion of schools resulted in significant changes of prevalences.

  • The resulting changes were small overall.

  • Even a reduced number of schools may be sufficient to gain valid estimates.

Abstract

Background

The European School Survey Project on Alcohol and Other Drugs (ESPAD) is a survey study that collects comparable data on substance use of students aged 15–16 years old in European countries. The present study aims at investigating the impact of school refusal to participate in ESPAD on substance use prevalence estimates.

Methods

Data came from the 2007 German ESPAD study; the sample consisted of 12,246 students in 552 schools within seven German federal states. A simulation approach was used in order to study the effects of systematic exclusion of participating schools on prevalence estimates of key ESPAD outcomes including the use of tobacco, alcohol, cannabis, and other illegal drugs.

Results

The systematic exclusion of schools based on city-, school-, and class size, school environment, and schools’ substance use policies resulted in significant changes in prevalence estimates in 23 of 25 examined combinations of selection criterion and outcome. Yet, these effects were small, with differences remaining below three percentage points around the original estimates.

Conclusions

This simulation approach suggests that nonparticipation of schools in surveys on students’ substance use in Germany does not largely affect the validity of resulting prevalence estimates. Even a reduced number of schools may be sufficient to gain valid prevalence figures.

Introduction

School surveys are frequently used to collect data on substance use in student populations. The European School Survey Project on Alcohol and Other Drugs (ESPAD) is a study that collects comparable data on substance use of students aged 15–16 years old in European countries. The survey was first carried out in 26 countries in 1995, while the most recent survey collected data from 36 countries in 2011 (Hibell et al., 2012).

In most ESPAD countries a large proportion of sampled schools took part in the surveys. In the 2011 data collection, the school participation rate was 90% or higher in 22 out of the 36 countries and the average for all countries was 85% (Hibell et al., 2012). However, the average has decreased over the years from about 95% in the 1995 and 1999 data collections, dropping to 91% in 2003, 89% in 2007, and 85% in 2011 (Hibell et al., 2004, Hibell et al., 2009, Hibell et al., 2012). In 2011, the lowest participation rate was reported for Great Britain (6%), and three more countries (Denmark, Germany, and Norway) reported rates below 50%. Germany is the country with the most dramatic drop in the proportion of participating schools from 91% in 2003 and 2007 to only 40% in 2011 (Hibell et al., 2004, Hibell et al., 2009, Hibell et al., 2012). This makes Germany an interesting country to study the potential effects of decreasing school participation on substance use prevalence estimates.

High response rates have long been considered a key criterion for epidemiological studies, but a general trend of declining response rates has been reported in recent decades (Galea & Tracy, 2007). The central concern associated with low response rates is that the resulting estimates may be affected by nonresponse bias, which is introduced when participant characteristics associated with the nonresponse are also related to the outcome examined. Therefore, the results of studies with a low response rate may not be valid.

The potential consequences of unit nonresponse have predominantly been examined in studies sampling individuals (Gerrits et al., 2001, Groves and Peytcheva, 2008, Keeter et al., 2000, Shahar et al., 1996) or organizations with respondents on the organizational level (e.g., sampling of schools, survey among principals) (Kano, Franke, Afifi, & Bourque, 2008). This holds also true for the area of substance abuse research, where studies examining the effects of nonresponse are, to the best of our knowledge, exclusively based on individuals (Kypri et al., 2004, Lahaut et al., 2002, Lahaut et al., 2003, Zhao et al., 2009). While some of these studies concluded that nonresponse bias is a significant problem (Lahaut et al., 2002, Zhao et al., 2009), others reported the opposite (Kypri et al., 2004). On the other hand, the effects of nonparticipation of institutions (e.g., schools) on substance use prevalence estimates derived from individual's responses nested within these institutions are unknown. Analysing data on school level provides the opportunity to examine the impact of nonparticipation related to structural, political, and social factors on the institutional rather than the individual level.

This study aimed at investigating the potential impact of school refusal to participate on substance use prevalence estimates based on 2007 ESPAD data collected in Germany. A simulation approach was used in order to study the effects of systematic exclusion of participating schools on the ESPAD key variables tobacco use, alcohol use, episodic heavy drinking (EHD), cannabis use, and use of illegal drugs other than cannabis.

The systematic selection and simulation of nonparticipation based on school characteristics were assumed to substantially influence substance use prevalences. For example, since we expected higher rates of substance use in larger cities (Tretter & Kraus, 2004), nonparticipation of schools from larger cities were expected to reduce prevalence estimates (Hypothesis 1). In addition, we explored if nonparticipation based on school or class size leads to substantial changes in substance use prevalence estimates.

Neighbourhood disadvantage may be related to adolescent substance use behaviours such as early initiation of substance use (Fite, Wynn, Lochman, & Wells, 2009) and a higher risk for cigarette, alcohol, or illegal drug use (Galea et al., 2004, Winstanley et al., 2008). Excluding schools from disadvantaged neighbourhoods was therefore hypothesized to lead to a decrease in prevalences (Hypothesis 2).

Lastly, school substance use policies and enforcement of these policies may be effective in reducing cigarette (Piontek, Bühler, & Kröger, 2007) or alcohol use (Evans-Whipp, Plenty, Catalano, Herrenkohl, & Toumbourou, 2013). We therefore expected that nonparticipation of schools with non-restrictive substance use policies to result in a reduction in prevalence estimates regarding cigarette and alcohol use (Hypothesis 3).

Section snippets

Procedure

Data from the 2007 ESPAD study in Germany were used (Kraus, Pabst, & Steiner, 2008). Seven out of 16 German federal states agreed to participate in the study, indicating a fairly good geographic representation of Germany: (1) Berlin as the capital within Brandenburg (2) as one of the three East-German states from the former German Democratic Republic, (3) Mecklenburg-Western Pomerania (northeast), (4) Thuringia (southeast), (5) Bavaria, a large rural state in the south, (6) Hesse, centrally

Sample description and correlations between school-level predictors

Sample characteristics of students and schools are summarized in Table 1. The highest prevalence was reported for past 30-day alcohol use (82.5%), followed by past 30-day EHD (60.1%), 30-day cigarette use (38.6%), 12-month cannabis use (17.7%), and lifetime use of illegal drugs other than cannabis (11.5%). Pairwise correlations of the school-level predictors are shown in Table 2. City, school, and class size were all significantly positively correlated with each other. Furthermore, city size

Discussion

The present study investigated the potential effects of school nonparticipation on prevalence estimates regarding the use of tobacco, alcohol, cannabis, and illegal drugs other than cannabis, using data from the German 2007 ESPAD study.

The simulation of systematic nonparticipation of schools resulted in significant changes of prevalence estimates in 23 out of 25 possible combinations of outcomes and selection variables. However, the changes in prevalence estimates were generally small. For

Limitations

The findings of the present study should be interpreted with several limitations in mind. Firstly, we conducted a simulation study based on a single cross-sectional study in seven German federal states. Results may not be generalizable to the rest of Germany, other countries, or other ESPAD study waves. Selection variables on the school level were correlated with each other and were not experimentally controlled; therefore, some associations between school selection variables and substance use

Conclusions

Systematic school-level nonresponse was associated with significant changes in prevalence estimates based on the 2007 German ESPAD study. However, our results suggest that these changes remain small, with the largest effects being increases in prevalence rates of approximately 2 pps. Overall, schools and school classes seem to represent the distribution of substance use among adolescents in the general population fairly adequately. Our results indicate that school nonparticipation in surveys

Conflict of interest

The authors report no conflicts of interest.

Acknowledgements

Funding for the project was provided by the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA): CC.13.SDI.021. The preparation of this manuscript was supported in part by the National Cancer Institute: NCI CA-113710.

References (29)

  • G. Gmel et al.

    Measuring alcohol consumption

    Contemporary Drug Problems

    (2004)
  • R.M. Groves et al.

    The impact of nonresponse rates on nonresponse bias a meta-analysis

    Public Opinion Quarterly

    (2008)
  • J. Hawkins et al.

    Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention

    Psychological Bulletin

    (1992)
  • B. Hibell et al.

    The ESPAD report 2003. Alcohol and other drug use among students in 35 European countries

    (2004)
  • View full text