Skip to main content
Log in

Response Effects in Surveys on Children and Adolescents: The Effect of Number of Response Options, Negative Wording, and Neutral Mid-Point

  • Published:
Quality and Quantity Aims and scope Submit manuscript

Abstract

Social researchers increasingly survey children and young adolescents.They are convinced that information about perspectives, attitudes, and behaviors of childrenshould be collected from the children themselves. Methodological expertise on surveying childrenis still scarce, and researchers rely on ad-hoc knowledge from fields such as child psychiatry and educationaltesting, or on methodological knowledge on surveying adults. Regarding adults, empirical evidenceshows that respondent characteristics (cognitive abilities) as well asquestion characteristics (question difficulty) affect response quality.

This study reports on a methodological survey experiment on theeffect of negatively formulated questions, the number of response options and offeringa neutral midpoint as response option question characteristics on the reliability of theresponses, using children and young adolescents as respondents.

The study shows no effects of negatively formulated questions onthe reliability measures, although children respond consistently differently on negativelyformulated questions than on positively formulated questions. Taking all results on the effectsof number of response options and offering a neutral midpoints on the different reliabilitymeasures into consideration; it would appear that offering about four responseoptions is optimal with children as respondents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alwin, D. F. & Krosnick, J. A. (1991). The reliability of survey attitude measurement. The influence of question and respondent attributes. Sociological Methods and Research 20(1): 139–181.

    Google Scholar 

  • Amato, P. R. & Ochiltree, G. (1987). Interviewing children about their families: A note on data quality. Journal of Marriage and the Family 49: 669–675.

    Google Scholar 

  • Andrews, F. M. (1984). Construct validity and error components of survey measures: A structural modeling approach. Public Opinion Quarterly 48: 409–442.

    Google Scholar 

  • Ayidiya, S. A. & McClendon, M. J. (1990). Response effects in mail surveys. Public Opinion Quarterly 54: 229–247.

    Google Scholar 

  • Benson, J. & Hocevar, D. (1985). The impact of item phrasing on the validity of attitude scales for elementary school children. Journal of Educational Measurement 22(3): 231–240.

    Google Scholar 

  • Biemer, P. P., Groves, R. M., Lyberg, L. E., Mathiowetz, N. A. & Sudman, S. (eds.) (1991). Measurement Errors in Surveys. New York: Wiley.

    Google Scholar 

  • Borgers, N. (1997). De invloed van taalvaardigheid op datakwaliteit bij vragenlijstonderzoek onder kinderen [in Dutch] (The Influence of Language and Reading Ability on Data Quality in Questionnaire Research with Children) (unpublished). Amsterdam: University of Amsterdam, Department of Education (POW).

  • Borgers, N. (1998). The Influence of Child and Question Characteristics on Item Non-Response and the Reliability in Self-Administered Questionnaires: Coding Scheme and Preliminary Results. Paper presented at the SMABS Conference, Leuven.

  • Borgers, N. & Hox, J. J. (1999). Item Non-Response in Questionnaire Research with Children. Paper presented at the International Conference on Survey Non-Response, Portland.

  • Borgers, N. & Hox, J. J. (2000, 3–6 October). Reliability of Responses in Questionnaire Research with Children. Paper presented at the Fifth International Conference on Logic and Methodology, Cologne, Germany.

  • Borgers, N., Leeuw, E. D. & Hox, J. J. (1999). Surveying children: Cognitive development and response quality in questionnaire research. In: A. Christianson, J. R. Gustafson, A. Klevmarken, B. Rosén, K.-G. Hansson & L. Granquist (eds.), Official Statistics in a Changing World. Stockholm: SCB, pp. 133–140.

    Google Scholar 

  • Borgers, N., Leeuw, E. D. & Hox, J. J. (2000). Children as respondents in survey research: Cognitive development and response quality. Bulletin de méthodologie sociologique (BMS) 66: 60–75.

    Google Scholar 

  • Bradburn, N. M. (1969). The Structure of Well-Being. Chicago: Aldine.

    Google Scholar 

  • Bradburn, N. M. & Caplovitz, D. (1965). Reports of Happiness. Chicago: Aldine.

    Google Scholar 

  • Bryk, A. S. & Raudenbush, S. W. (1992). Hierarchical Linear Model: Applications and Data Analysis Methods. Newbury Park: Sage.

    Google Scholar 

  • Cannel, Miller & Oksenberg. (1990). Research on Interviewing Techniques. Field Experiment in Health Research 1971–1977.

  • De Leeuw, E. D. & Otter, M. E. (1995). The reliability of children's responses to questionnaire items; Question effects in children's questionnaire data. In: J. J. Hox, B. F. V. D. Meulen, J. M. A. M. Janssens, J. J. F. T. Laak & L.W. C. Tavecchio (eds.), Advances in Family Research. Amsterdam: Thesis Publishers.

    Google Scholar 

  • Felix, J. & Sikkel, D. (1999). Attrition bias in telepanel research. Kwantitatieve Methoden 20(61): 101–110.

    Google Scholar 

  • Flavell, J. H. (1985). Cognitive Development, 2nd edn. Englewood Cliffs, New Jersey: Prentice-Hall International, Inc.

    Google Scholar 

  • Goldstein, H. (1995). Multilevel Statistical Models, 2nd edn. London: Edward Arnold.

    Google Scholar 

  • Groves, R. (1989). Survey Error and Survey Costs. New York: Wiley.

    Google Scholar 

  • Hayes, W. L. (1974). Statistics for Social Sciences, 2nd edn. London: Holt, Rinehart and Winston.

    Google Scholar 

  • Holaday, B. & Turner-Henson, A. (1989). Response effects in surveys with school-age children. Nursing Research 38(4): 248–250.

    Google Scholar 

  • Hox, J. J. (1986). Het gebruik van hulptheorieen bij operationalisering. Een studie rond het begrip subjectief welbevinden. Unpublished PhD, University of Amsterdam, Amsterdam.

  • Hox, J. J. (2002, forthcoming). Multilevel Analysis of Regression and Structural Equation Models. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Knäper, B., Belli, R. F., Hill, D. H. & Herzog, A. R. (1997). Question difficulty and respondents' cognitive ability: The effect of data quality. Journal of Official Statistics: An International Review 13(2).

  • Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5: 213–236.

    Google Scholar 

  • Krosnick, J. A. & Alwin, D. F. (1987). An evaluation of cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly 51(2): 201–219.

    Google Scholar 

  • Krosnick, J. A. & Fabrigar, L. R. (1997). Designing Rating Scaling for Effective Measurement in Surveys. Survey Measurement and Process Quality. New York: Wiley.

    Google Scholar 

  • Krosnick, J. A., Narayan, S. & Smith, W. R. (1996). Satisficing in surveys: Initial evidence. New Directions for Evaluation 70: 29–43.

    Google Scholar 

  • Leeuw, E. D. de, Hox, J. J. & Snijkers, G. (1998). The effect of computer-assisted interviewing on data quality. A review. In: B. Blyth (ed.), Market Research and Information Technology. ESOMAR Monograph. Amsterdam: Esomar, pp. 173–198.

    Google Scholar 

  • Linden, F. J. V. D., Dijkman, T. & Roeders, P. J. B. (1983). Metingen van kenmerken van het persoonssysteem en sociale systeem. Nijmegen: Hoogveld Instituut.

    Google Scholar 

  • Lyberg, L., Biemer, P., Collins, M., Leeuw, E. D., Dippo, C., Schwarz, N. & Trewin, D. (eds.) (1997). Survey Measurement and Process Quality, Vol. 1. New York: Wiley.

    Google Scholar 

  • Marsh, H. W. (1986). Negative item bias in rating scales for preadolescent children: A cognitive-developmental phenomenon. Developmental Psychology 22(1): 37–49.

    Google Scholar 

  • Narayan, S. & Krosnick, J. A. (1996). Education moderates some response effects in attitude measurement. Public Opinion Quarterly 60: 58–88.

    Google Scholar 

  • Otter, M., Mellenberg, D. & Glopper, K. D. (1995). The relation between information-processing variables and test-retest stability for questionnaire items. Journal of Educational Measurement 32(2): 199–216.

    Google Scholar 

  • Otter, M. E. (1993). Leesvaardigheid, leesonderwijs en buitenschools lezen. Instrumentatie en effecten. Unpublished Ph.D., University of Amsterdam, Amsterdam.

  • Piaget, J. (1929). Introduction to the Child's Conception of the World. New York: Harcourt.

    Google Scholar 

  • Piaget, J. (1955). The Language and Thought of the Child. New York: Meridian Books.

    Google Scholar 

  • Raaijmakers, Q. A. W., Van Hoof, A., 't Hart, H., Verbogt, T. F. M. A. & Vollebergh, W. A. M. (2000). Adolescents' midpoint responses on Liker-type scale items: Neutral or missing values? International Journal of Public Opinion Research 12(2).

  • Raudenbush, S. W. (1994). Random effects models. In: C. H. & L. V. Hedges (eds.), The Handbook of Research Synthesis. New York: Russell Sage Foundation.

    Google Scholar 

  • Rodgers, W. L., Andrews, F. M. & Herzog, A. R. (1989). Quality of Survey Measures: A Structural Modeling Approach.

  • Rodgers, W. L., Herzog, A. R. & Andrews, F. M. (1988). Interviewing older adults: Validity of self-reports of satisfaction. Psychology and Aging 3(1): 264–272.

    Google Scholar 

  • Rosenberg, M. (1965). Society and the Adolescents Self Image. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Rosenberg, M. (1979). Conceiving the Self. New York: Basic.

    Google Scholar 

  • Schwarz, N. & Hippler, H. J. (1995). Subsequent questions may influence answers to preceding questions in mail surveys. Public Opinion Quarterly 59: 93–97.

    Google Scholar 

  • Schwarz, N. & Knäuper, B. (1999). Cognitive aging and accuracy of self-report. In: D. Park & N. Schwarz (eds.), Cognitive Aging: A Primer. Philadelphia, PA: Psychology Press, pp. 233–252.

    Google Scholar 

  • Schwarz, N., Knäuper, B. & Park, D. (1998). Aging, Cognition, and Context Effects: How Differential Context Effects Invite Misleading Conclusions about Cohort Differences. Paper presented at the Conference on Methodological Issues in Official Statistics, Stockholm.

  • Schwarz, N., Park, D., Knäuper, B. & Sudman, S. (1998). Cognition, Aging, and Self-Reports. Philadelphia, PA: Psychology Press.

    Google Scholar 

  • Scott, J. (1997). Children as respondents: Methods for improving data quality. In: L. Lyberg (ed.), Survey Measurements and Process Quality. New York: Wiley.

    Google Scholar 

  • Sudman, S., Bradburn, N. M. & Schwarz, N. (1996). Thinking about Answers. The Application of Cognitive Processes to Survey Methodology. San Fransisco: Josey-Bass.

    Google Scholar 

  • Tourangeau, R. & Rasinski, K. A. (1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin 103: 299–314.

    Google Scholar 

  • Vaillancourt, P. M. (1973). Stability of children's survey responses. Public Opinion Quarterly 373–387.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Dirk Sikkel.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Borgers, N., Sikkel, D. & Hox, J. Response Effects in Surveys on Children and Adolescents: The Effect of Number of Response Options, Negative Wording, and Neutral Mid-Point. Quality & Quantity 38, 17–33 (2004). https://doi.org/10.1023/B:QUQU.0000013236.29205.a6

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:QUQU.0000013236.29205.a6

Navigation