Abstract
Does the manifestation of response set effects in World-Wide Web (WWW) questionnaire responses differ from that in paper-and-pencil (PP) questionnaire responses? Two thousand six hundred and forty-nine volunteers responded to a WWW questionnaire, and 458 volunteers responded to a PP questionnaire consisting of five personality scales. Five response sets were examined. For four of them—random response, item nonresponse, extreme response, and acquiescentresponse—no statistically significant differences between PP and WWW data were found. The PP administration elicited a statistically higher number of errors (uncodable responses), although η2 was less than .02. An analysis of interitem and interscale correlations did not differentiate WWW and PP data. Data from the Marlowe-Crowne Social Desirability Scale, the Perfectionist Self-Presentation Scale, and the Computer Anxiety Scale did not differentiate WWW and PP data in terms of either means or scale reliabilities. It was concluded that WWW data may be comparable to PP data and that the WWW is a potentially useful and valid data collection tool.
Article PDF
Similar content being viewed by others
References
Beach, D. A. (1989). Identifying the random responder.Journal of Psychology,123, 101–103.
Cronbach, L. J. (1946). Response sets and test validity.Educational & Psychological Measurement,6, 475–494.
Helgeson, J. G., &Ursic, M. L. (1989). The decision process equivalency of electronic versus pencil-and-paper data collection methods.Social Science Computer Review,7, 296–310.
Hewitt, P., &Flett, G. (1995).Perfectionistic Self-Presentation Scale, Unpublished test, York University, Toronto.
Kiesler, S., &Sproull, L. S. (1986). Response effects in the electronic survey.Public Opinion Quarterly,50, 402–413.
Krantz, J. H., Ballard, J., &Scher, J. (1997). Comparing the results of laboratory and World-Wide Web samples on the determinants of female attractiveness.Behavior Research Methods, Instruments, & Computers,29, 264–269.
Locke, S. D., &Gilbert, B. O. (1995). Method of psychological assessment, self-disclosure, and experiential differences: A study of computer, questionnaire, and interview assessment formats.Journal of Social Behaviour & Personality,10, 255–263.
Loyd, B. H., &Gressard, C. (1984). Reliability and factorial validity of computer attitude scales.Educational & Psychological Measurement,44, 501–505.
Marlowe, D., &Crowne, D. P. (1960). A new scale of social desirability independent of psychopathology.Journal of Consulting Psychology,24, 349–354.
Martin, K. L., &Nagao, D. H. (1989). Some effects of computerized interviewing on job applicant responses.Journal of Applied Psychology,74, 72–80.
Pasveer, K. A., &Ellard, J. H. (1998). The making of apersonality inventory: Help from the WWW.Behavior Research Methods, Instruments, & Computers,30, 309–313.
Pettit, F. A. (1998).Pettit WWW Random Responding Scale, Unpublished test, York University, North York, ON.
Pettit, F. A. (1999). Exploring the use of the World Wide Web as a psychology data collection tool.Computers in Human Behavior,15, 67–71.
Schmidt, W. C. (1997). World-Wide Web survey research: Benefits, potential problems, and solutions.Behavior Research Methods, Instruments, & Computers,29, 274–279.
Smith, M. A., &Leigh, G. (1997). Virtual subjects: Using the Internet as an alternative source of subjects and research environment.Behavior Research Methods, Instruments, & Computers,29, 496–505.
Sproull, L. S. (1986). Using electronic mail for data collection in organizational research.Academy of Management Journal,29, 159–169.
Webster, J., &Compeau, D. (1996). Computer-assisted versus paper-and-pencil administration of questionnaires.Behavior Resea rch Methods, Instruments, & Computers,28, 567–576.
Welch, N., &Krantz, J.H. (1996). The World-Wide Web as a medium for psychoacoustical demonstrations and experiments: Experience and results.Behavior Research Methods, Instruments, & Computers,28, 192–196.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pettit, F.A. A comparison of World-Wide Web and paper-and-pencil personality questionnaires. Behavior Research Methods, Instruments, & Computers 34, 50–54 (2002). https://doi.org/10.3758/BF03195423
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03195423