Abstract
Internet research is appealing because it is a cost- and time-efficient way to access a large number of participants; however, the validity of Internet research for important subjective well-being (SWB) surveys has not been adequately assessed. The goal of the present study was to validate the Satisfaction With Life Scale (SWLS; Diener, Emmons, Larsen, & Griffin, 1985), the Positive and Negative Affect Schedule (PANAS-X; Watson & Clark, 1994), and the Subjective Happiness Scale (SHS; Lyubomirsky & Lepper, 1999) for use on the Internet. This study compared the quality of data collected using paper-based (paper-and-pencil version in a lab setting), computer-based (Web-based version in a lab setting), and Internet (Web-based version on a computer of the participant’s choosing) surveys for these three measures of SWB. The paper-based and computer-based experiment recruited two college student samples; the Internet experiments recruited a college student sample and an adult sample responding to ads on different social-networking Web sites. This study provides support for the reliability, validity, and generalizability of the Internet format of the SWLS, PANAS-X, and SHS. Across the three experiments, the results indicate that the computer-based and Internet surveys had means, standard deviations, reliabilities, and factor structures that were similar to those of the paper-based versions. The discussion examines the difficulty of higher attrition for the Internet version, the need to examine reverse-coded items in the future, and the possibility that unhappy individuals are more likely to participate in Internet surveys of SWB.
Article PDF
Similar content being viewed by others
References
Bech, M., & Kristensen, M. B. (2009). Differential response rates in postal and Web-based surveys in older respondents. Survey Research Methods, 3, 1–6.
Birnbaum, M. H. (2004). Human research and data collection via the Internet. Annual Review of Psychology, 55, 803–832.
Bliven, B. D., Kaufman, S. E., & Spertus, J. A. (2001). Electronic collection of health-related quality of life data: Validity, time, benefits, and patient preference. Quality of Life Research, 10, 15–21.
Brecko, B. N., & Carstens, R. (2006). Online data collection in SITES 2006. Paper versus Web survey—Do they provide comparable results? The Second IEA International Research Conference: Proceedings of the IRC-2006, 2, 261–269.
Buchanan, T. (2002). Online assessment: Desirable or dangerous? Professional Psychology: Research & Practice, 33, 148–154.
Buchanan, T., Johnson, J. A., & Goldberg, L. R. (2005). Implementing a five-factor personality inventory for use on the Internet. European Journal of Psychological Assessment, 21, 115–127.
Buchanan, T., & Smith, J. L. (1999). Using the Internet for psychological research: Personality testing on the World Wide Web. British Journal of Psychology, 90, 125–144.
Carlbring, P., Brunt, S., Bohman, S., Austin, D., Richards, J., Ost, L.-G., & Andersson, G. (2007). Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Computers in Human Behavior, 23, 1421–1434.
Chuah, S. C., Drasgow, F., & Roberts, B. W. (2006). Personality measurement: Does the medium matter? No. Journal of Research in Personality, 40, 359–376.
Cole, M. S., Bedeian, A. G., & Field, H. S. (2006). The measurement equivalence of Web-based and paper-and-pencil measures of transformational leadership. Organizational Research Methods, 9, 339–368.
Davis, R. N. (1999). Web-based administration of a personality questionnaire: Comparison with traditional methods. Behavior Research Methods, Instruments, & Computers, 31, 572–577.
DeNeve, K. M., & Cooper, H. (1998). The happy personality: A metaanalysis of 137 personality traits and subjective well-being. Psychological Bulletin, 124, 197–229.
Denscombe, M. (2008). The length of responses to open-ended questions: A comparison of online and paper questionnaires in terms of a mode effect. Social Science Computer Review, 26, 359–368.
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction With Life Scale. Journal of Personality Assessment, 49, 71–75.
Diener, E., & Seligman, M. E. P. (2004). Beyond money. Toward an economy of well-being. Psychological Science in the Public Interest, 5, 1–31.
Feldt, L. S. (1969). A test of the hypothesis that Cronbach’s alpha or Kuder-Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 363–373.
Fortson, B. L., Scotti, J. R., Del Ben, K. S., & Chen, Y.-C. (2006). Reliability and validity of an Internet traumatic stress survey with a college student sample. Journal of Traumatic Stress, 19, 709–720.
George, C. E., Lankford, J. S., & Wilson, S. E. (1992). The effects of computerized versus paper-and-pencil administration on measures of negative affect. Computers in Human Behavior, 8, 203–209.
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust Web-based studies: A comparative analysis of six preconceptions about Internet questionnaires. American Psychologist, 59, 93–104.
Herrero, J., & Meneses, J. (2006). Short Web-based versions of the perceived stress (PSS) and Center for Epidemiological Studies-Depression (CESD) Scales: A comparison to pencil and paper responses among Internet users. Computers in Human Behavior, 22, 830–846.
Hewson, C., & Charlton, J. P. (2005). Measuring health beliefs on the Internet: A comparison of paper and Internet administrations of the Multidimensional Health Locus of Control Scale. Behavior Research Methods, 37, 691–702.
Im, E.-O., Chee, W., Bender, M., Cheng, C.-Y., Tsai, H.-M., Kang, N. M., & Lee, H. (2005). The psychometric properties of pen-andpencil and Internet versions of the Midlife Women’s Symptom Index (MSI). International Journal of Nursing Studies, 42, 167–177.
John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The Big Five inventory—Versions 4a and 54. Berkeley: University of California, Berkeley, Institute of Personality and Social Research.
Joinson, A. (1999). Social desirability, anonymity, and Internet-based questionnaires. Behavior Research Methods, Instruments, & Computers, 31, 433–438.
Kim, S., & Feldt, L. S. (2008). A comparison of tests for equality of two or more independent alpha coefficients. Journal of Educational Measurement, 45, 179–193.
Kveton, P., Jelínek, M., Klimusová, H., & Vobořil, D. (2007). Data collection on the Internet: Evaluation of Web-based questionnaires. Studia Psychologica, 49, 81–88.
Lonsdale, C., Hodge, K., & Rose, E. A. (2006). Pixels vs. paper: Comparing online and traditional survey methods in sport psychology. Journal of Sport & Exercise Psychology, 28, 100–108.
Luthans, F., Avey, J. B., & Patera, J. L. (2008). Experimental analysis of a Web-based training intervention on the learning and development of positive psychological states. Academy of Management Learning & Education, 7, 209–221.
Lyubomirsky, S., & Lepper, H. S. (1999). A measure of subjective happiness: Preliminary reliability and construct validation. Social Indicators Research, 46, 137–155.
McGraw, K. O., Tew, M. D., & Williams, J. E. (2000). The integrity of Web-delivered experiments: Can you trust the data? Psychological Science, 11, 502–506.
McNeil, J. M., & Fleeson, W. (2006). The causal effect of extraversion on positive affect and neuroticism on negative affect: Manipulating state extraversion and state neuroticism in an experimental approach. Journal of Research in Personality, 40, 529–550.
Nosen, E., & Woody, S. R. (2008). Online surveys: Effect of research design decision on rates of invalid participation and data credibility. Graduate Student Journal of Psychology, 10, 3–14.
Pavot, W., & Diener, E. (1993). Review of the Satisfaction With Life Scale. Psychological Assessment, 5, 164–172.
Reimers, S., & Stewart, N. (2007). Adobe Flash as a medium for online experimentation: A test of reaction time measurement capabilities. Behavior Research Methods, 39, 365–370.
Riva, G., Teruzzi, T., & Anolli, L. (2003). The use of the Internet in psychological research: Comparison of online and offline questionnaires. CyberPsychology & Behavior, 6, 73–80.
Rosenfeld, P., Giacalone, R. A., Knouse, S. B., Doherty, L. M., Vicino, S. M., Kantor, J., & Greaves, J. (1991). Impression management, candor, and microcomputer-based organizational surveys: An individual differences approach. Computers in Human Behavior, 7, 23–32.
Ryan, R. M., & Deci, E. L. (2001). On happiness and human potentials: A review of research on hedonic and eudaimonic well-being. Annual Review of Psychology, 52, 141–166.
Saucier, G. (1994). Mini-Markers: A brief version of Goldberg’s unipolar Big-Five markers. Journal of Personality Assessment, 63, 506–516.
Schimmack, U., Oishi, S., Furr, R. M., & Funder, D. C. (2004). Personality and life satisfaction: A facet-level analysis. Personality & Social Psychology Bulletin, 30, 1062–1075.
Shih, T.-H., & Fan, X. (2008). Comparing response rates from Web and mail surveys: A meta-analysis. Field Methods, 20, 249–271.
Steel, P., Schmidt, J., & Shultz, J. (2008). Refining the relationship between personality and subjective well-being. Psychological Bulletin, 134, 138–161.
Tabachnick, B. G., & Fidell, L. S. (1989). Using multivariate statistics (2nd ed.). New York: Harper & Row.
Tseng, H.-M., Macleod, H. A., & Wright, P. (1997). Computer anxiety and measurement of mood change. Computers in Human Behavior, 13, 305–316.
Vadillo, M. A., Bárcena, R., & Matute, H. (2006). The Internet as a research tool in the study of associative learning: An example from overshadowing. Behavioural Processes, 73, 36–40.
Vallejo, M. A., Mañanes, G., Comeche, M. I., & Díaz, M. I. (2007). Comparison between administration via Internet and paper-andpencil administration of two clinical instruments: SCL-90-R and GHQ-28. Journal of Behavior Therapy & Experimental Psychiatry, 39, 201–208.
Vereecken, C. A., & Maes, L. (2006). Comparison of a computer-administered and paper-and-pencil-administered questionnaire on health and lifestyle behaviors. Journal of Adolescent Health, 38, 426–432.
Vittersø, J., & Nilsen, F. (2002). The conceptual and relational structure of subjective well-being, neuroticism, and extraversion: Once again, neuroticism is the important predictor of happiness. Social Indicators Research, 57, 89–118.
Watson, D., & Clark, L. A. (1994). The PANAS-X: Manual for the Positive and Negative Affect Schedule—Expanded Form. Iowa City: University of Iowa, Department of Psychology.
Williams, J. E., McGraw, K. O., & Tew, M. D. (1999). Undergraduate labs and computers: The case for PsychExps. Behavior Research Methods, Instruments, & Computers, 31, 287–291.
Yu, S.-C., & Yu, M.-N. (2007). Comparison of Internet-based and paper-based questionnaires in Taiwan using multisample invariance approach. CyberPsychology & Behavior, 10, 501–507.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Howell, R.T., Rodzon, K.S., Kurai, M. et al. A validation of well-being and happiness surveys for administration via the Internet. Behavior Research Methods 42, 775–784 (2010). https://doi.org/10.3758/BRM.42.3.775
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BRM.42.3.775