Elsevier

Social Science & Medicine

Volume 62, Issue 4, February 2006, Pages 1009-1021
Social Science & Medicine

Lies, damned lies and statistics? Reliability and personal accounts of smoking among young people

https://doi.org/10.1016/j.socscimed.2005.07.002Get rights and content

Abstract

Smoking remains a major problem among young people in Europe. However, within the research community examining the issue, debate continues about the best way of assessing the extent of that problem. Questions have been raised about the extent to which existing techniques for generating statistical representations of patterns of youth smoking can address a range of problems connected with identifying, accounting for and correcting unreliable self-report smoking data. Using empirical data from the UK Liverpool Longitudinal Smoking Study (LLSS), this paper argues that self-report measures of smoking, treated in isolation from participants’ personal accounts, can disguise problems with the reliability and validity of a given study. Using longitudinal qualitative and quantitative data in dialogue, two main factors contributing to unreliable data are discussed: (a) participants’ access to and familiarity with frameworks of everyday cultural knowledge about the practice of smoking, and (b) participants’ retrospective revision of events in line with their current goals, aspirations and self-understandings. The conclusion drawn is that research has to employ multiple methods, minimally incorporating some personal contribution from participants, to explore the complex character of the problem of smoking and to avoid the difficulties posed by the models of smoking behaviour embodied within stand-alone statistical research.

Introduction

Research suggests that smoking rates among young people stabilised in Europe in the 1990s, with ‘little decline’ (Griesbach, Amos, & Currie, 2003, p. 41). However, increases were observed in some countries, such as Austria, Belgium, Germany, Scotland and Wales, particularly among teenage girls (Wold, Holstein, Griesbach, & Currie, 2000). These shifts in the demography of smoking, towards women and the young, are indicative of countries entering the third and fourth stages in the evolution of the tobacco epidemic (Lopez, Collishaw, & Piha, 1994), and reference the need for research that is able to engage with changing tobacco control contexts and the emergence, in parallel, of distinct vulnerable groups with different intervention needs. Smoking among the young is ‘a critical indicator of the initiation of tobacco use and harbinger of future trends in the prevalence of tobacco dependence among adults’ (Thun & da Costa e Silva, 2003, p. 9), and hence, of both future tobacco-related morbidity and mortality (Doll, Peto, Boreham, & Sutherland, 2004) as well as the demand for the health-care services to deal with tobacco use (Foulds & Godfrey, 1995). This makes the accurate and reliable monitoring of smoking among young people a key component of tobacco control, public health and the ‘fight against tobacco’ (Wolfson, 2001).

In the British context, however, the adequacy of the ‘surveillance systems’ (Thun & da Costa e Silva, 2003, p. 7) in place to monitor smoking among the young has been questioned. In their second report on ‘The Tobacco Industry and the Health Risks of Smoking’, the House of Commons Health Select Committee, highlighted important gaps in our knowledge and the urgency of research into ‘the age at which children start smoking, [and] the reasons they begin, continue and quit smoking’ (House of Commons Health Select Committee, 2000).

Drawing on results from the Liverpool Longitudinal Smoking Study (LLSS), a study of smoking in a single birth cohort of children in Liverpool, the argument put forward here is that this call for a comprehensive evidence-base for tobacco control policy in the UK represents something of a problem for a research community that is still looking for the best way to study smoking among the young. Picking up on the difference between, on the one hand, measuring how many young people start smoking and at what age, and, on the other, exploring why young people start at the ages they do, this paper distinguishes two broad, yet interrelated research strategies;

  • (a)

    macro-level statistical approaches: broadly quantitative approaches which employ variable analysis, and sampling theory, to link smoking behaviour with a range of demographic, personality and socio-cultural variables.

  • (b)

    micro-level biographical approaches: broadly qualitative approaches which track individual biographies and the role that smoking plays within them through participants’ personal accounts.

Applying insights into individual cases, gained from pursuing the second strategy, to the statistical trends identified through pursuit of the first, this paper argues that statistical approaches will typically (i) over-estimate prevalence and experimentation among young children, (ii) under-estimate experimentation and prevalence rates among older children and adolescents and (iii) distort the complex patterns of initiation, maintenance and quitting that characterise smoking among young people as a whole.

Section snippets

Statistical misrepresentations?

Problems with the accuracy of smoking prevalence statistics are widely recognised. For instance, research has shown that adopting different definitions of smoking status results in different prevalence rates for the same populations (Henriksen & Jackson, 1999; Thun & da Costa e Silva, 2003). Prevalence rates are, therefore, reflections or artefacts of the methods used to produce them as much as they are reflections of the true rate of prevalence in a population (Babor, Brown, & DelBoca, 1990).

Method

In order to understand why participants routinely gave contradictory reports of their own smoking behaviour, a strategy was developed for checking the consistency of participants’ responses, which draws on and expands the scheme proposed by Henriksen and Jackson (1999). This strategy consisted of a number of steps for methodically cataloguing the response-histories gathered in the course of the study.

  • (1)

    A comprehensive tobacco-use history was developed for each participant, based on annual

Overview

Using the strategy outlined above it was possible to identify three different types of inconsistent response histories. Those where inconsistencies were:

  • Concentrated at the beginning of the study's timeframe, when the children were aged 5–7.

  • Seen across the entire timeframe of the study.

  • Concentrated in the study's later period, when the children were making the transition from childhood to adolescence, ages 11–13.

The patterns of inconsistency within these age groups were quite different.

Discussion

One of the assumptions that has guided much research into patterns of smoking behaviour among children, whether qualitative or quantitative, is the idea that participants’ accounts of their own smoking behaviour directly report on events within their lives. That is: someone has either smoked or they have not; and this is something that remains true of that individual independently of what subsequently happens to them. The findings from our study suggest that this assumption is not warranted.

Conclusion

In the course of this paper, the lack of an accurate statistical measure of smoking prevalence within the tobacco research literature has been highlighted. Using empirical data, the paper has attempted to show that such measures are systematically unattainable. Arguing that cotinine testing and its equivalents are too time-specific for the detailed analysis of smoking behaviour in childhood and adolescence, it was noted that the alternative, longitudinal studies, while providing more detail,

Acknowledgements

We would like to thank the Roy Castle Foundation for funding this research. We would also like to thank all of the Primary and Secondary Schools in Liverpool who have taken part in the study, as well as the Liverpool and Sefton Local Education Authorities and Health Education Boards. We would like to acknowledge our gratitude to the children, and the parents of the children, for their continued support. Finally, we would like to thank Prof. Jane Springett, of John Moores University, and Dr.

References (29)

  • House of Commons Select Committee on Health. (2000). Second report on The tobacco industry and the health risks of...
  • C. Jackson et al.

    A longitudinal study predicting patterns of cigarette smoking in late childhood

    Health Education and Behaviour

    (1998)
  • T. Johnson et al.

    The reliability of self-reported age on onset of tobacco, alcohol and illicit drug use

    Addiction

    (2001)
  • D. Kenkel et al.

    Smoke or fog? The usefulness of retrospectively reported information about smoking

    Addiction

    (2003)
  • Cited by (17)

    • Health surveillance through social networks

      2015, Social Networks
      Citation Excerpt :

      For example, self-reports have led to inconsistent smoking prevalence rates within a population (Henriksen and Christine, 1999) or under-reporting of sensitive health behaviors (Kenkel et al., 2003). To increase the validity of self-reports, some have turned to longitudinal methods to check for errors and inconsistencies (Henriksen and Christine, 1999; Johnson and Mott, 2001; Mair et al., 2006), while others have turned to biological testing (Vartiainen et al., 2002). Regardless of the effectiveness of these methods, they all tend to bear a large cost.

    • Age of smoking milestones: Longitudinal inconsistencies and recanting

      2015, Journal of Adolescent Health
      Citation Excerpt :

      Respondents may reclassify an experience on the basis new experiences. For example, current smokers may in retrospect revise what they regard as their first cigarette to when they first inhaled deeply [24]. However, Add Health uses clear and behaviorally specific language to reduce interpretation confusion.

    • Reduced prepulse inhibition in adolescents at risk for psychosis: A 2-year follow-up study

      2011, Journal of Psychiatry and Neuroscience
      Citation Excerpt :

      Despite this limitation, our results demonstrated that unmedicated individuals did not differ from medicated individuals and that group results did not change when medicated individuals were excluded from the analysis. A possible limitation of the current study is that smoking status was based on self report, which cannot exclude the possibility that unreliable data were submitted.34 Nicotine intake can increase PPI levels in controls and patients with schizophrenia35–37 and consequently may have interfered with our outcome.

    • Ethnomethodology’s Legacies and Prospects

      2022, The Ethnomethodology Program: Legacies and Prospects
    • Accuracy of Retrospective Reports of Family Environment

      2018, Journal of Child and Family Studies
    View all citing articles on Scopus
    View full text