Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Can’t We Make It Any Shorter?

The Limits of Personality Assessment and Ways to Overcome Them

Published Online:January 01, 2014https://doi.org/10.1027/1614-0001/a000141

Abstract

Psychological constructs are becoming increasingly important in social surveys. Scales for the assessment of these constructs are usually developed primarily for individual assessment and decision-making. Hence, in order to guarantee high levels of reliability, measurement precision, and validity, these scales are in most cases much too long to be applied in surveys. Such settings call for extremely short measures validated for the population as a whole. However, despite the unquestionable demand, appropriate measures are still lacking. There are several reasons for this. In particular, short scales have often been criticized for their potential psychometric shortcomings with regard to reliability and validity. In this article, the authors discuss the advantages of short scales as alternative measures in large-scale surveys. Possible reasons for the assumed limited psychometric qualities of short scales will be highlighted. The authors show that commonly used reliability estimators are not always appropriate for judging the quality of scales with a minimal number of items, and they offer recommendations for alternative estimation methods and suggestions for the construction of a thorough short scale.

References

  • Abell, N. , Springer, D. W. , Kamata, A. (2009). Developing and validating rapid assessment instruments. New York, NY: Oxford University Press. First citation in articleCrossref, Google Scholar

  • Allison, P. J. , Guichard, C. , Fung, K. , Gilain, L. (2003). Dispositional optimism predicts survival status 1 year after diagnosis in head and neck cancer patients. Journal of Clinical Oncology, 21, 543–548. First citation in articleCrossref, Google Scholar

  • Arthur, W. Jr. , Graziano, W. G. (1996). The five-factor model, conscientiousness, and driving accident involvement. Journal of Personality, 64, 593–618. First citation in articleCrossref, Google Scholar

  • Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman. First citation in articleGoogle Scholar

  • Baumert, A. , Beierlein, C. , Schmitt, M. , Kemper, C. J. , Kovaleva, A. , Liebig, S. , Rammstedt, B. (2013). Measuring four facets of Justice Sensitivity with two items each. Journal of Personality Assessment, 1996, 380–390. doi: 10.1080/00223891.2013.836526 First citation in articleGoogle Scholar

  • Beckmann, J. F. , Guthke, J. , Jäger, A. O. , Süß, H.-M. , Beauducel, A. (1999). Berliner Intelligenzstruktur-Test (BIS), Form 4 [Berlin Intelligence Structure Test, Form 4]. Diagnostica, 45, 55–61. First citation in articleLink, Google Scholar

  • Beierlein, C. , Davidov, E. , Behr, D. , Cieciuch, J. , László, Z. , Schwartz, S. (2014). Construction and validation of the survey compatible Schwartz Values Short Scale 4 using international samples. Manuscript in preparation. First citation in articleGoogle Scholar

  • Beierlein, C. , Kemper, C. J. , Kovaleva, A. , Rammstedt, B. (2012a). Ein Messinstrument zur Erfassung politischer Kompetenz- und Einflusserwartungen: Political Efficacy Kurzskala (PEKS) [A measurement instrument for assessing political competence and control expectations: Political Efficacy Short Scale (PEKS)]. GESIS Working Papers 2012|18. Köln, Germany: GESIS. First citation in articleGoogle Scholar

  • Beierlein, C. , Kemper, C. J. , Kovaleva, A. , Rammstedt, B. (2012b). Kurzskala zur Messung des zwischenmenschlichen Vertrauens: Die Kurzskala Interpersonales Vertrauen (KUSIV3) [Short scale for assessing interpersonal trust: The short scale interpersonal trust (KUSIV3)]. Köln, Germany: GESIS GESIS Working Papers 2012|22. First citation in articleGoogle Scholar

  • Beierlein, C. , Kemper, C. J. , Kovaleva, A. , Rammstedt, B. (2013). Kurzskala zur Erfassung allgemeiner Selbstwirksamkeitserwartungen (ASKU) [Short scale for assessing general self-efficacy expectations (ASKU)]. Methoden, Daten, Analysen, 7, 251–278. First citation in articleGoogle Scholar

  • Borghans, L. , Duckworth, A. L. , Heckman, J. J. , Weel, B. T. (2008). The economics and psychology of personality traits. Journal of Human Resources, 43, 972–1059. First citation in articleCrossref, Google Scholar

  • Boyle, G. J. (1991). Does item homogeneity indicate internal consistency or tern redundancy in psychometric scales? Personality and Individual Differences, 12, 291–294. First citation in articleCrossref, Google Scholar

  • Burisch, M. (1984). Approaches to personality inventory construction: A comparison of merits. American Psychologist, 39, 214–227. First citation in articleCrossref, Google Scholar

  • Butcher, J. N. , Dahlstrom, W. G. , Graham, J. R. , Tellegen, A. M. , Kaemmer, B. , Engel, R. (2000). Minnesota Multiphasic Personality Inventory (MMPI-2). Göttingen, Germany: Hogrefe. First citation in articleGoogle Scholar

  • Caprara, G. V. , Vecchione, M. , Capanna, C. , Mebane, M. (2009). Perceived political self‐efficacy: Theory, assessment, and applications. European Journal of Social Psychology, 39, 1002–1020. First citation in articleCrossref, Google Scholar

  • Clark, L. A. , Watson, D. (1995). Constructing validity: Basic issued in objective scale development. Psychological Assessment, 7, 309–319. First citation in articleCrossref, Google Scholar

  • Cohen, R. J. , Swerdlik, M. E. (2005). Psychological testing and assessment: An introduction to tests and measurement. Boston, MA: McGraw-Hill Publishing. First citation in articleGoogle Scholar

  • Costa, P. T. , McCrae, R. R. (1992). Revised NEO Personality Inventory and NEO Five Factor Professional Manual. Odessa, FL: Psychological Assessment Resources. First citation in articleGoogle Scholar

  • Credé, M. , Harms, P. , Niehorster, S. , Gaye-Valentine, A. (2012). An evaluation of the consequences of using short measures of the big five personality traits. Journal of Personality and Social Psychology, 102, 874–888. First citation in articleCrossref, Google Scholar

  • Cronbach, L. J. , Shavelson, R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64, 391–418. First citation in articleCrossref, Google Scholar

  • Diener, E. D. , Emmons, R. A. , Larsen, R. J. , Griffin, S. (1985). The satisfaction with life scale. Journal of Personality Assessment, 49, 71–75. First citation in articleCrossref, Google Scholar

  • Dillman, D. A. (2007). Mail and Internet surveys: The tailored design method (2nd ed.). New York, NY: Wiley. First citation in articleGoogle Scholar

  • Edwards, P. , Roberts, I. , Sandercock, P. , Frost, C. (2004). Follow-up by mail in clinical trials. Does questionnaire length matter? Controlled Clinical Trials, 25, 31–52. First citation in articleCrossref, Google Scholar

  • Fishman, J. A. , Galguera, T. (2003). Introduction to test construction in the social and behavioral sciences: A practical guide. Ranham, MD: Rowman & Littlefield. First citation in articleGoogle Scholar

  • German Data Forum . (2010). Building on progress: Expanding the research infrastructure for the social, economic, and behavioral sciences. Opladen, Germany: Budrich UniPress. First citation in articleGoogle Scholar

  • Goldberg, L. R. (2005). Why personality measures should be included in epidemiological surveys: A brief commentary and a reading list. Eugene, OR: Oregon Research Institute. First citation in articleGoogle Scholar

  • Gosling, S. D. , Rentfrow, P. J. , Swann, W. B. (2003). A very brief measure of the Big-Five personality domains. Journal of Research in Personality, 37, 504–528. First citation in articleCrossref, Google Scholar

  • Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24, 79–132. First citation in articleCrossref, Google Scholar

  • Gottfredson, L. S. , Deary, I. J. (2004). Intelligence predicts health and longevity, but why? Current Directions in Psychological Science, 13, 1–4. First citation in articleCrossref, Google Scholar

  • Graham, J. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66, 930–944. First citation in articleCrossref, Google Scholar

  • Heise, D. R. (1969). Separating reliability and stability in test-retest correlation. American Sociological Review, 34, 93–101. First citation in articleCrossref, Google Scholar

  • Hoyle, R. H. , Stephenson, M. T. , Palmgreen, P. , Lorch, E. P. , Donohew, R. L. (2002). Reliability and validity of a brief measure of sensation seeking. Personality and Individual Differences, 32, 401–414. First citation in articleCrossref, Google Scholar

  • Jonason, P. K. , Webster, G. D. (2010). The dirty dozen: A concise measure of the dark triad. Psychological Assessment, 22, 420. First citation in articleCrossref, Google Scholar

  • Kaplan, R. , Saccuzzo, D. (2009). Psychological testing: Principles, applications, and issues. Belmont, CA: Wadsworth, Cengage Learning. First citation in articleGoogle Scholar

  • Kemper, C. J. , Beierlein, C. , Bensch, D. , Kovaleva, A. , Rammstedt, B. (2012). Eine Kurzskala zur Erfassung des Gamma-Faktors sozial erwünschten Antwortverhaltens: Die Kurzskala Soziale Erwünschtheit-Gamma (KSE-G) [A short scale for assessing the gamma-factor of social desirable response behavior: The short scale Social Desirability-Gamma (KSE-G)]. GESIS Working Papers 2012|25. Köln, Germany: GESIS. First citation in articleGoogle Scholar

  • Kemper, C. J. , Beierlein, C. , Kovaleva, A. , Rammstedt, B. (2012). Eine Kurzskala zur Messung von Optimismus-Pessimismus: Die Skala Optimismus-Pessimismus-2 (SOP2) [A Short Scale for Assessing Optimism-Pessimism: The Optimism-Pessimism-2 Scale (SOP2)]. GESIS Working Papers 2012|15. Köln, Germany: GESIS. First citation in articleGoogle Scholar

  • Kemper, C. J. , Beierlein, C. , Kovaleva, A. , Rammstedt, B. (2013). Entwicklung und Validierung einer ultrakurzen Operationalisierung des Konstrukts Optimismus-Pessimismus–Die Skala Optimismus-Pessimismus-2 (SOP2) [Development and validation of an ultra-short assessment of optimism-pessimism–The Optimism-Pessimism-2 Scale (SOP2)]. Diagnostica, 59, 119–129. First citation in articleLink, Google Scholar

  • Kemper, C. J. , Brähler, E. , Zenger, M. (2013). Psychologische und sozialwissenschaftliche Kurzskalen: Standardisierte Erhebungsinstrumente für Wissenschaft und Praxis [Psychological and social science short scales: Standardized instruments for research and practice]. Berlin, Germany: Medizinisch-Wissenschaftliche Verlagsgesellschaft. First citation in articleGoogle Scholar

  • Kemper, C. J. , Lutz, J. , Neuser, J. (2011). Konstruktion und Validierung einer Kurzform der Skala Angst vor negativer Bewertung (SANB-5) [Construction and Validation of a scale for assessing fear of negative appraisal]. Klinische Diagnostik und Evaluation, 4, 343–360. First citation in articleGoogle Scholar

  • Kersting, M. (2006). Zur Beurteilung der Qualität von Tests: Resümee und Neubeginn [The evaluation of test quality: Summary and new beginning]. Psychologische Rundschau, 57, 243–253. First citation in articleLink, Google Scholar

  • Kovaleva, A. , Beierlein, C. , Kemper, C. J. , Rammstedt, B. (2012a). Eine Kurzskala zur Messung von Kontrollüberzeugung: Die Skala Internale-Externale-Kontrollüberzeugung-4 (IE-4) [A short scale for assessing control beliefs: The scale Internal-External Control Beliefs-4 (IE-4)]. GESIS Working Papers 2012|19. Köln, Germany: GESIS. First citation in articleGoogle Scholar

  • Kovaleva, A. , Beierlein, C. , Kemper, C. J. , Rammstedt, B. (2012b). Eine Kurzskala zur Messung von Impulsivität nach dem UPPS-Ansatz: Die Skala Impulsives-Verhalten-8 (I-8) [A short scale for assessing impulsivity following the UPPS approach]. GESIS Working Papers 2012|20. Köln, Germany: GESIS. First citation in articleGoogle Scholar

  • Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537–567. First citation in articleCrossref, Google Scholar

  • Kruyen, P. M. (2012). Using Short Tests and Questionnaires for Making Decisions about Individuals: when is Short Too Short? (Doctoral dissertation, Tilburg University). Retrieved from: arno.uvt.nl/show.cgi?fid=128226 First citation in articleGoogle Scholar

  • Kruyen, P. M. , Emons, W. H. M. , Sijtsma, K. (2013). On the shortcomings of shortened tests: A literature review. International Journal of Testing, 13, 223–248. First citation in articleCrossref, Google Scholar

  • Loevinger, J. (1954). The attenuation paradox in test theory. Psychological Bulletin, 51, 493–504. First citation in articleCrossref, Google Scholar

  • Lohmann, H. , Spieß, C. K. , Groh-Samberg, O. , Schupp, J. (2009). Analysepotenziale des sozio-oekonomischen Panels (SOEP) für die empirische Bildungsforschung [Analytic potential of the German Socio-economic Panel for educational research]. Zeitschrift für Erziehungswissenschaft, 12, 252–280. First citation in articleCrossref, Google Scholar

  • Lord, F. M. , Novick, M. R. (1968). Statistical theory of mental test scores. Reading, MA: Addison-Wesley. First citation in articleGoogle Scholar

  • Lutz, J. , Kemper, C. J. , Beierlein, C. , Margraf-Stiksrud, J. , Rammstedt, B. (2013). Konstruktion und Validierung einer Skala zur relativen Messung von physischer Attraktivität mit einem Item: Das Attraktivitätsrating 1 (AR1) [Construction and validation of a scale for the relative assessment of of physical attractivity using a single item: The attractivity rating 1(AR1)]. Methoden, Daten, Analysen, 7, 209–232. First citation in articleGoogle Scholar

  • Marteau, T. M. , Bekker, H. (1992). The development of a six‐item short‐form of the state scale of the Spielberger State-Trait Anxiety Inventory (STAI). British Journal of Clinical Psychology, 31, 301–306. First citation in articleCrossref, Google Scholar

  • McDonald, R. P. (1999). Test theory. A unified treatment. Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Mondak, J. J. (2010). Personality and the Foundations of Political Behavior. New York, NY: Cambridge University Press. First citation in articleCrossref, Google Scholar

  • Niemi, R. G. , Carmines, E. G. , McIver, J. P. (1986). The impact of scale length on reliability and validity. Quality and Quantity, 20, 371–376. First citation in articleCrossref, Google Scholar

  • Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill. First citation in articleGoogle Scholar

  • Nunnally, J. C. , Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill. First citation in articleGoogle Scholar

  • Petermann, F. (Ed.). (2012). Wechsler Adult Intelligence Sale-Fourth Edition (WAIS-IV) Deutsche Version [German Version]. Frankfurt, Germany: Pearson Assessment. First citation in articleGoogle Scholar

  • Raes, F. , Pommier, E. , Neff, K. D. , Van Gucht, D. (2011). Construction and factorial validation of a short form of the self‐compassion scale. Clinical Psychology & Psychotherapy, 18, 250–255. First citation in articleCrossref, Google Scholar

  • Rammstedt, B. (2010). Subjective indicators. In German Data Forum. (Eds.), Building on progress. Expanding the research infrastructure for the social, economic, and behavioral sciences (pp. 813–824). Opladen: Budrich UniPress. First citation in articleGoogle Scholar

  • Rammstedt, B. , John, O. P. (2007). Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German. Journal of Research in Personality, 41, 203–212. First citation in articleCrossref, Google Scholar

  • Rammstedt, B. , Kemper, C. J. , Schupp, J. (Eds.). (2013). Standardisierte Kurzskalen zur Erfassung psychologischer Merkmale in Umfragen. [Standardized short-scale measures for assessing psychological constructs in surveys]. Methoden, Daten, Analysen, 7, 145–152. First citation in articleGoogle Scholar

  • Rammstedt, B. , Kemper, C. J. , Klein, M. C. , Beierlein, C. , Kovaleva, A. (2013). Eine kurze Skala zur Messung der fünf Dimensionen der Persönlichkeit: 10 Item Big Five Inventory (BFI-10) [A short scale for assessing the Big Five dimensions of personality (BFI-10)]. Methoden, Daten, Analyse, 7, 235–251. First citation in articleGoogle Scholar

  • Rammstedt, B. , Spinath, F. M. (2013). Öffentliche Datensätze und ihr Mehrwert für die psychologische Forschung [Publicly available data sets and their use for psychological research]. Psychologische Rundschau, 64, 101–102. First citation in articleLink, Google Scholar

  • Rasmussen, H. N. , Scheier, M. F. , Greenhouse, J. B. (2009). Optimism and physical health: A meta-analytic review. Annals of Behavioral Medicine, 37, 239–256. First citation in articleCrossref, Google Scholar

  • Raykov, T. , Marcoulides, G. A. (2011). Introduction to psychometric theory. New York, NY: Taylor & Francis. First citation in articleCrossref, Google Scholar

  • Robins, R. W. , Hendin, H. M. , Trzesniewski, K. H. (2001). Measuring global self-esteem: Construct validation of a single-item measure and the Rosenberg Self-Esteem Scale. Personality and Social Psychology Bulletin, 27, 151–161. First citation in articleCrossref, Google Scholar

  • Schermelleh-Engel, K. , Werner, C. S. (2012). Methoden der Reliabilitätsbestimmung [Methods of estimating reliability]. In H. Moosbrugger, A. Kelava (Eds.) Testtheorie und Fragebogenkonstruktion [Test and questionnaire construction]. (2nd ed.). (pp. 119–142). Heidelberg, Germany: Springer. First citation in articleCrossref, Google Scholar

  • Schipolowski, S. , Wilhelm, O. , Schroeders, U. , Kovaleva, A. , Kemper, C. J. , Rammstedt, B. (2013). BEFKI GC-K: Eine Kurzskala zur Messung kristalliner Intelligenz [BEFKI GC-K: A short scale for the measurement of crystallized intelligence]. Methoden, Daten, Analysen, 7, 155–183. First citation in articleGoogle Scholar

  • Schmidt, F. L. , Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274. First citation in articleCrossref, Google Scholar

  • Schoen, H. , Steinbrecher, M. (2013). Beyond total effects: Exploring the interplay of personality and attitudes in affecting turnout in the 2009 German federal election. Political Psychology, 34, 533–552. First citation in articleCrossref, Google Scholar

  • Schupp, J. , Spieß, C. K. , Wagner, G. G. (2008). Die verhaltenswissenschaftliche Weiterentwicklung des Erhebungsprogramms des SOEP [The behavioral research enhancements of the SOEP survey programme]. Vierteljahreshefte zur Wirtschaftsforschung, 77, 63–76. First citation in articleCrossref, Google Scholar

  • Schwartz, S. H. , Boehnke, K. (2004). Evaluating the structure of human values with confirmatory factor analysis. Journal of Research in Personality, 38, 230–255. First citation in articleCrossref, Google Scholar

  • Schwartz, S. H. , Cieciuch, J. , Vecchione, M. , Davidov, E. , Fischer, R. , Beierlein, C. , Konty, M. (2012). Refining the theory of basic individual values: New concepts and measurements. Journal of Personality and Social Psychology, 103, 663–688. First citation in articleCrossref, Google Scholar

  • Schweizer, K. (2011). Some thoughts concerning the recent shift from measures with many items to measures with few items. European Journal of Psychological Assessment, 27, 71–72. doi: 10.1027/1015-5759/a000056 First citation in articleLink, Google Scholar

  • Siegrist, J. , Wege, N. , Pühlhofer, F. , Wahrendorf, M. (2009). A short generic measure of work stress in the era of globalization: Effort-reward imbalance. International Archives of Occupational and Environmental Health, 82, 1005–1013. First citation in articleCrossref, Google Scholar

  • Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107–120. First citation in articleCrossref, Google Scholar

  • Smith, G. T. , McCarthy, D. M. , Anderson, K. G. (2000). On the sins of short form development. Psychological Assessment, 12, 102–111. First citation in articleCrossref, Google Scholar

  • Strenze, T. (2007). Intelligence and socioeconomic success: A meta-analytic review of longitudinal research. Intelligence, 35, 401–426. First citation in articleCrossref, Google Scholar

  • Stumpf, H. , Angleitner, A. , Wieck, T. , Jackson, D. N. , Beloch-Till, H. (1985). Deutsche Personality Research Form (PRF). Handanweisung (Manual). Göttingen, Germany: Hogrefe. First citation in articleGoogle Scholar

  • Thalmayer, A. G. , Saucier, G. , Eigenhuis, A. (2011). Comparative validity of brief- to medium-length Big Five and Big Six questionnaires. Psychological Assessment, 23, 995–1009. First citation in articleCrossref, Google Scholar

  • Tourangeau, R. , Rips, L. J. , Rasinski, K. (2000). The psychology of survey response. Cambridge, UK: Cambridge University Press. First citation in articleCrossref, Google Scholar

  • Traub, R. (1997). Classical Test Theory in historical perspective. Educational Measurement: Issues and Practice, 16, 8–14. First citation in articleCrossref, Google Scholar

  • Wanous, J. P. , Reichers, A. E. , Hudy, M. J. (1997). Overall job satisfaction: How good are single-item measures? Journal of Applied Psychology, 82, 247–252. First citation in articleCrossref, Google Scholar

  • Ware, J. E. Jr. , Kosinski, M. , Keller, S. D. (1996). A 12-Item Short-Form Health Survey: Construction of scales and preliminary tests of reliability and validity. Medical Care, 34, 220–233. First citation in articleCrossref, Google Scholar

  • Zinbarg, R. E. , Revelle, W. , Yovel, I. , Li, W. (2005). Cronbach’s Alpha, Revelle’s Beta, McDonald’s Omega: Their relations with each and two alternative conceptualizations of reliability. Psychometrika, 70, 123–133. First citation in articleCrossref, Google Scholar