Are we keeping the people who used to stay? Changes in correlates of panel survey attrition over time

Abstract

As survey response rates decline, correlates of survey participation may also be changing. Panel studies provide an opportunity to study a rich set of correlates of panel attrition over time. We look at changes in attrition rates in the American National Election Studies from 1964 to 2004, a repeated panel survey with a two-wave pre-post election design implemented over multiple decades. We examine changes in attrition rates by three groups of variables: sociodemographic and ecological characteristics of the respondent and household, party affiliation and political and social attitudes recorded at the first interview, and paradata about the first wave interview. We find relatively little overall change in the pre-post election panel attrition rates, but important changes in demographic correlates of panel attrition over time. We also examine contact and cooperation rates from 1988 to 2004.

Introduction

A panel study timed around a political election is one common method of conducting an election study. Election studies are conducted to understand who votes and why they vote, and take place in many countries around the world, including the United States, Canada, countries throughout the European Union, Australia and Japan (ANES, 2008). A typical election study asks a random sample of voting age adults about their political and social attitudes immediately before an election. Soon after the election, these pre-election survey respondents are once again asked about their political and social attitudes and – importantly – their voting behavior. These surveys are referred to as pre- and post-election surveys.

Inevitably, some people who participated in the pre-election survey do not participate in the post-election survey. Attrition, or nonresponse to wave two or later of a panel survey, reduces sample size for analyses and biases survey estimates when those who participate are systematically different from those who do not (Bethlehem, 2002, Deming, 1953, Groves and Couper, 1998). Demographic characteristics have been shown to be related to panel attrition, with older people, minorities and males generally more likely to attrite (Aneshensel et al., 1989, Burkam and Lee, 1998, Fitzgerald et al., 1998, Gray et al., 1996, Loosveldt et al., 2002, Lynn et al., 2005, MaCurdy et al., 1998, Mirowsky and Reynolds, 2000, Peracchi, 2002, Sharot, 1991, Watson and Wooden, 2009). Persons with more education and higher income are less likely to attrite (Fitzgerald et al., 1998, Gray et al., 1996, Loosveldt et al., 2002, Lynn et al., 2005, MaCurdy et al., 1998, Mirowsky and Reynolds, 2000, Watson and Wooden, 2009), and persons living in urban areas are more likely to attrite (Gray et al., 1996, Lynn et al., 2005, Watson and Wooden, 2009). Attitudinal measures have also been examined as predictors of panel attrition, including social (Waterton and Lievesley, 1987) and political attitudes (Lepkowski and Couper, 2002, Loosveldt and Carton, 1997, Loosveldt et al., 2002). Nonrespondents in election studies tend to be those who are less politically interested or involved in politics (Brehm, 1993, Burden, 2000, Voogt and Saris, 2003; Voogt, 2005).

Different from cross-sectional surveys, sample units to later waves of panel surveys already have experienced at least one interview. As such, characteristics of the previous survey interview also predict panel attrition. Item nonresponse to questions in a previous wave of the study (Burkam and Lee, 1998, Lepkowski and Couper, 2002, Loosveldt and Carton, 1997, Loosveldt et al., 2002, Watson and Wooden, 2009) is associated with higher attrition rates, while the longer interviews in prior waves are associated with higher retention rates (Bogen, 1996).

Although declines in response rates for cross-sectional household surveys over the last four decades are well documented (Curtin et al., 2005, de Leeuw and de Heer, 2002, Steeh, 1981), changes in panel survey attrition rates over these years have been less frequently examined, with limited evidence for declining trends (Agency for Healthcare Research, 2008, Atrostic et al., 2001, Westat, 1998). More importantly, unknown is whether the kinds of people who participate in surveys are changing. To our knowledge, no research has been conducted on whether these factors consistently predict attrition from year to year, holding the wave constant, that is, from wave 1 to wave 2. In other words, are the types of nonrespondents now different from those from a few decades ago?

Among those social changes with the potential to affect who participates in surveys, women’s labor force participation in the US increased substantially with more women working outside the home (Blau, 1998, Dye, 2005, US Census Bureau, 2008). There has been a general aging of the US population (Hobbs and Stoops, 2002), accompanied by an increase in education levels (Stoops, 2004). The racial/ethnic composition of the US also became increasingly non-white (Hobbs and Stoops, 2002). Where people live also changed, with a shift back to urban centers, especially in the 1990s and early 2000s (Mackun, 2005). Some of these changes over time can be attributed to historical events spurring social change, others to birth cohort differences, and others to within-individual changes over time (Alwin and McCammon, 2003, Fienberg and Mason, 1979). As the public changes, their views on the value of surveys or time that can be devoted to surveys might change. We hypothesize that these social groups (as defined by sex, race, age, education, and urbanicity) are the most likely to exhibit changes in participation rates over time.

In an election study, participation may be particularly attractive to respondents who are affiliated with certain political parties. We hypothesize that persons whose political party wins a Presidential election will be more likely to cooperate with the post-election survey request. Thus, as political power changes over time, so will cooperation rates. These differences may not be reflected in overall retention rates as we do not expect differences in contact rates over time for different political parties.

This paper examines changes in correlates of post-election survey participation in the American National Election Studies from 1964 to 2004. We also examine whether there are differences in correlates of making contact with the sampled person and cooperation, conditional on contact, between 1988 and 2004. These analyses permit us to understand whether there is change in characteristics of nonrespondents, and thus the potential for change in nonresponse bias of survey estimates, over these years.