Skip to main content
Log in

Using ad hoc measures for response styles: a cautionary note

  • Research Note
  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

This research note addresses the challenge of how to optimally measure acquiescence response style (ARS) and extreme response style (ERS). This is of crucial importance in assessing results from studies that have tried to identify antecedents of response styles (such as age, education level, national culture). Using survey data from the Netherlands, a comparison is made between the traditional method and a more recently proposed method of measuring ARS and ERS (i.e., the convergent validity across both methods is assessed). The traditional method is based on an ad hoc set of related items. The alternative method uses a set of randomly sampled items to optimize heterogeneity and representativeness of the items. It is found that the traditional method may lead to response style measures that are suboptimal for estimating levels of ARS and ERS as well as relations of ARS and ERS with other variables (like hypothesized antecedents). Recommendations on how to measure response styles are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arce-Ferrer A.J.: An investigation into the factors influencing extreme-reponse style: improving meaning of translated and culturally adapted rating scales. Educ. Psychol. Meas. 66, 374–392 (2006)

    Article  Google Scholar 

  • Baumgartner H., Steenkamp J.-B.E.M.: Response styles in marketing research: a cross-national investigation. J. Mark. Res. 36, 143–156 (2001)

    Article  Google Scholar 

  • Baumgartner H., Steenkamp J.-B.E.M.: Response biases in marketing research. In: Grover, R., Vriens , M.(eds) The Handbook of Marketing Research: Uses, Misuses, and Future Advances, pp. 95–109. Sage, Thousand Oaks, CA (2006)

  • Bearden W.O., Netemeyer R.G.: Handbook of Marketing Scales: Multi-item Measures for Marketing and Consumer Behavior Research, 2nd edn. Sage, Thousand Oaks, CA (1999)

    Google Scholar 

  • Billiet J.B., Davidov E.: Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociol. Methods Res. 36, 542–562 (2008)

    Article  Google Scholar 

  • Billiet J.B., McClendon MK.J.: Modeling acquiescence in measurement models for two balanced sets of items. Struct. Equ. Model. 7, 608–628 (2000)

    Article  Google Scholar 

  • Bruner G.C., James K.E., Hensel P.J.: Marketing Scales Handbook. A Compilation of Multi-item Measures. American Marketing Association (AMA), Chicago, IL (2001)

    Google Scholar 

  • Cheung G.W., Rensvold R.B.: Assessing extreme and acquiescence response sets in cross-cultural research. J. Cross-Cult. Psychol. 31, 187–212 (2000)

    Article  Google Scholar 

  • De Jong M.G., Steenkamp J.-B.E.M., Fox J.-P., Baumgartner H.: Using item response theory to measure extreme response style in marketing research: a global investigation. J. Mark. Res. 45, 104–115 (2008)

    Article  Google Scholar 

  • Forsman L.: Giving extreme response to items in self-esteem scales: response set or personality trait? Eur. J. Psychol. Assess. 9, 33–40 (1993)

    Google Scholar 

  • Franzblau A.: A Primer of Statistics for Non-Statisticians. Harcourt, Brace & World, New York (1958)

    Google Scholar 

  • Greenleaf E.A.: Improving rating scale measures by detecting and correcting bias components in some response styles. J. Mark. Res. 29, 176–188 (1992a)

    Article  Google Scholar 

  • Greenleaf E.A.: Measuring extreme response style. Public Opin. Q. 56, 328–351 (1992b)

    Article  Google Scholar 

  • Grunert, K.G., Brunso, K., Bisp, S.: Food-related lifestyle: development of a cross-culturally valid instrument for market surveillance, MAPP Working Paper no. 12. Aarhus School of Business, Aarhus, Denmark (1993)

  • Hinkle D.E., Wiersma W., Jurs S.G.: Applied Statistics for the Behavioral Sciences, 2nd edn. Houghton Mifflin College, Boston (1988)

    Google Scholar 

  • Harzing A.-W.: Response styles in cross-national survey research: A 26-country study. Int. J. Cross-Cult. Manag. 6, 243–266 (2006)

    Article  Google Scholar 

  • Hofstede G.: Culture’s Consequences. International Differences in Work-Related Values. Sage, London (1980)

    Google Scholar 

  • Hofstede G.: Culture’s Consequences. Comparing Values Behaviors, Institutions and Organizations, 2nd edn. Sage, Thousand Oaks, CA (2001)

    Google Scholar 

  • House R.J., Hanges P.J., Javidan M., Dorfman P.W., Gupta V.: Culture, Leadership, and Organizations. The Globe Study of 62 Societies. Sage, Thousand Oaks, CA (2004)

    Google Scholar 

  • Johnson T., Kulesa P., Cho Y.I., Shavitt S.: The relation between culture and response styles: evidence from 19 countries. J. Cross-Cult. Psychol. 36, 264–277 (2005)

    Article  Google Scholar 

  • Krosnick J.A.: Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl. Cogn. Psychol. 5, 213–236 (1991)

    Article  Google Scholar 

  • Krosnick J.A.: Survey methodology. Ann. Rev. Psychol. 50, 537–567 (1999)

    Article  Google Scholar 

  • Krosnick, J.A.: The measurement of attitudes. In: Albarracin, D., Johnson, B.T., Zanna , M.P. The Handbook of Attitudes, pp. 21–76. Lawrence Erlbaum Associates, Mahwah, NJ (2005)

  • Laros, F.J.M.: Emotions in consumer research: an application to novel food products. Dissertation, Tilburg University, The Netherlands (2006)

  • Marin G., Gamba R.J., Marin B.V.: Extreme response style and acquiescence among Hispanics. J. Cross-Cult. Psychol. 23, 498–509 (1992)

    Article  Google Scholar 

  • Narayan S., Krosnick J.A.: Education moderates some response effects in attitude measurement. Public Opin. Q. 60, 58–88 (1996)

    Article  Google Scholar 

  • O’Sullivan C., Scholderer J., Cowan C.: Measurement equivalence of the food related lifestyle instrument (FRL) in Ireland and Great Britain. Food Qual. Prefer. 16, 1–12 (2005)

    Article  Google Scholar 

  • Paulhus D.L.: Measurement and control of response bias. In: Robinson, J.P., Shaver, P.R., Wrightsman, L.S.(eds) Measures of Personality and Social Psychological Attitudes, vol. 1, Academic Press, San Diego, CA (1991)

    Google Scholar 

  • Reynolds N.L., Simintiras A.C., Diamantopoulos A.: Theoretical justification of sampling choices in international marketing research: key issues and guidelines for researchers. J. Int. Bus. Stud. 34, 80–89 (2003)

    Article  Google Scholar 

  • Robinson J.P., Shaver P.R., Wrightsman L.S.: Measures of Personality and Social Psychological Attitudes, vol. 1. Academic Press, San Diego, CA (1991)

    Google Scholar 

  • Ross C.E., Mirowsky J.: Socially-desirable response style and acquiescence in a cross-cultural survey of mental health. J. Health Soc. Behav. 25, 189–197 (1984)

    Article  Google Scholar 

  • Schimmack U., Böckenholt U., Reisenzein R.: Response styles in affect ratings: making a mountain out of a molehill. J. Pers. Assess. 78, 461–483 (2002)

    Article  Google Scholar 

  • Scholderer J., Brunsø K., Bredahl L., Grunert K.G.: Cross-cultural validity of the food-related lifestyles instrument (FRL) within Western Europe. Appetite 42, 197–211 (2004)

    Article  Google Scholar 

  • Schuman H., Presser S.: Questions and Answers in the Attitude Surveys: Experiments on Question Form, Wording, and Context. Harcourt Brace Jovanovich, New York, NY (1981)

    Google Scholar 

  • Smith P.B.: Acquiescent response bias as an aspect of cultural communication style. J. Cross-Cult. Psychol. 35, 50–61 (2004)

    Article  Google Scholar 

  • Van Herk H., Poortinga Y.H., Verhallen T.M.M.: Response styles in rating scales: evidence of method bias in data from six EU countries. J. Cross-Cult. Psychol. 35, 346–360 (2004)

    Article  Google Scholar 

  • Warnecke R.B., Johnson T.P., Chávez N., Sudman S., O’Rourke D.P., Lacey L., Horm J.: Improving question wording in surveys of culturally diverse populations. Ann. Epidemiol. 7, 334–342 (1997)

    Article  Google Scholar 

  • Watson D.: Correcting for acquiescent response bias in the absence of a balanced scale: an application to class consiousness. Sociol. Methods Res. 21, 52–88 (1992)

    Article  Google Scholar 

  • Weijters, B.: Response styles in consumer research. Dissertation, Vlerick Leuven Gent Management School, Belgium (2006).

  • Weijters B., Schillewaert N., Geuens M.: Assessing response styles across modes of data collection. J. Acad. Mark. Sci. 36, 409–422 (2008)

    Article  Google Scholar 

  • Winkler J.D., Kanouse D.E., Ware J.E. Jr: Controlling for acquiescence response set in scale development. J. Appl. Psychol. 67, 555–561 (1982)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alain De Beuckelaer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

De Beuckelaer, A., Weijters, B. & Rutten, A. Using ad hoc measures for response styles: a cautionary note. Qual Quant 44, 761–775 (2010). https://doi.org/10.1007/s11135-009-9225-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-009-9225-z

Keywords

Navigation