Equivalence of paper and pencil vs Internet forms of the ZKPQ-50-CC in Spanish and French samples
Introduction
From the mid 1990s onwards, researchers in psychology have given personality tests on the Internet, using the items of traditional paper and pencil administered inventories. There is currently a large scientific literature on-line versus paper and pencil answer formats in psychological tests (Buchanan, 2000, Buchanan, 2001, Buchanan, 2002, Buchanan et al., 2005, Buchanan and Smith, 1999, Hertel et al., 2002, Vazire, 2006) The advantages of the on-line-administration have been generally deemed as being of low cost and access to a greater number of subjects in much less time. In addition, data entry requirements are largely eliminated, and the potential for errors in data entry is largely reduced (Barak and English, 2002, Buchanan, 2006).
Most of the authors conclude that the on-line formats of personality questionnaires have good equivalence and similar psychometric properties to the traditional paper and pencil forms (Chuah et al., 2006, Davis, 1999, Gosling et al., 2004). However, equivalence between test formats needs to be established (American Educational Research Association, 1999). As noted by Chuah et al. (2006), it cannot be assumed that an Internet form has a similar validity and reliability as the paper and pencil form and its psychometric properties must be compared to those of the paper-and-pencil format. If there are mean differences in scales it is important to have norms for each format.
Psychologists have previously compared the equivalence of the two types of formats (Ferrando & Lorenzo-Seva, 2005), that have been approached from statistics relying on classical test theory (CTT), or from statistics relying on item response theory (IRT), although most works have been based on the former. Besides, CTT is based in the comparison of the following statistics in the two formats PP–OL: (a) the means and standard deviations, (b) scale internal consistency, and (c) factor structure with exploratory factor analysis or structural equation modeling (Buchanan and Smith, 1999, Steinberg et al., 2000). Mean differences, reliabilities and factor analysis can support the equivalence of Internet and paper-and-pencil administrations of personality tests (including ITR) (Chuah et al., 2006).
The Zuckerman–Kuhlman Personality Questionnaire (ZKPQ; Zuckerman, 2002, Zuckerman et al., 1993) was derived empirically through exploratory factor analysis of a number of personality questionnaires (Zuckerman et al., 1988, Zuckerman et al., 1991). The ZKPQ is based on a five-factor psychobiological model (Zuckerman, 2005). Aluja et al. (2006) developed a short version of the ZKPQ (ZKPQ-50-CC) selecting 10 items per scale: Impulsive Sensation Seeking (ImpSS), Neuroticism–Anxiety (N–Anx), Aggression–Hostility (Agg–Host), Activity (Act), and Sociability (Sy). This short version has equivalent factor structure and reliability in four language versions: English, French, German, and Spanish (Aluja et al., 2006). The ZKPQ-50-CC predicts scores on the Personality Disorders measured by the Millon Clinical Multiaxial Inventory-III. The predictive power was the same as that for the NEO-FFI-R (Aluja, García, Cuevas, & García, 2007). This ZKPQ version is for testing situations where it is necessary to use short protocols. Also, the abbreviated form can be used to obtain data through the Internet. For this reason, it is important to know if the on-line data and paper and pencil format have equivalent characteristics in similar samples.
The aim of this study was to check the equivalence between the paper and pencil and Internet formats of the ZKPQ-50-CC, comparing their psychometric properties. The properties compared between the two formats of administration were: (a) means and standard deviations of scales, (b) distribution measures: Skewness and kurtosis, (c) internal consistency alpha, (d) factorial structure and congruence coefficients inter-format, and (e) confirmatory factorial analysis for each format and multi-group analysis.
Section snippets
Subjects
The sample of this study consisted in 5972 subjects from Spain and Switzerland (1856 males [31.1%], and 4116 females [68.9%]). Inclusion in the study was restricted to subjects under 30 years of age in order to reduce the age differences between samples and ZKPQ-50-CC formats. The age means differences were significant, but the effect size was small. The sample was divided in two groups by country according to answer format: on-line (OL) or paper-and-pencil (PP). Spanish subjects included 3827
Format descriptive statistics and alpha comparisons by country
Table 1, Table 2 show means, deviations, kurtosis, skewness and alpha, and Student-Fisher t-tests for significance of OL vs PP formats and effect sizes for these differences for Spanish (Table 1) and Swiss (Table 2) on-line and paper and pencil ZKPQ-50-CC formats. For the Spanish version the means differences for OL–PP formats were statistically significant for all scales, except for Agg–Host. Significant differences were obtained between the SD’s of Sy and Act [F (1940, 1885) = 25.66, p < .001; [F
Discussion
This study was designed to assess the equivalence of the paper-and-pencil and on-line Internet formats of the ZKPQ-50-CC. Data collection through the Internet is very useful, even for the investigation using personality questionnaires due to the advantages commented on in the introductory section (Buchanan, 2000, Buchanan, 2001, Buchanan, 2002, Hertel et al., 2002, Vazire, 2006). To demonstrate the equivalence among the two formats, new perspectives in personality research might be explored,
References (31)
- et al.
A cross-cultural shortened form of the ZKPQ (ZKPQ-50-CC) adapted to English, French, German and Spanish languages
Personality an Individual Difference
(2006) - et al.
Personality assessment: Does the medium matter? No
Journal of Research in Personality
(2006) Informant reports: A cheap, fast, and easy method for personality assessment
Journal of Research in Personality
(2006)- et al.
Five (or three) robust questionnaire scales factors of personality without culture
Personality and Individual Differences
(1991) - et al.
The MCMI-III personality disorders scores predicted by the NEO-FFI-R and the ZKPQ-50-CC: A comparative study
Journal of Personality Disorders
(2007) - et al.
Standards for educational and psychological testing
(1999) Amos 4.01
(1999)- et al.
Prospects and limitations of psychological testing on the Internet
Journal of Technology in Human Services
(2002) - et al.
Behavioral characteristics of Internet addicted students in college
Chinese Journal of Clinical Psychology
(2005) Potential of the Internet for personality research
Online personality assessment
Online assessment: Desirable or dangerous?
Professional Psychology: Research and Practice
Personality testing on the Internet: What we know, and what we do not
Nonequivalence of on-line and paper-and-pencil psychological test: The case of the prospective memory questionnaire
Behavior Research Methods
Using the Internet for psychological research: Personality testing on the World Wide Web
British Journal of Psychology
Cited by (24)
The resilient, the restraint and the restless: Personality types based on the Alternative Five-Factor Model
2018, Personality and Individual DifferencesCitation Excerpt :The ZKPQ-50-CC was administered together with other questionnaires, in both paper and online form. Previous studies have provided support for the equivalence of the paper and online forms of the ZKPQ-50-CC (Aluja, Rossier, & Zuckerman, 2007), thus supporting this dual way of gathering data. The ZKPQ-50-CC is a shortened, cross-cultural version of the Zuckerman-Kuhlman Personality Questionnaire (ZKPQ; Zuckerman, Kuhlman, Joireman, Teta, & Kraft, 1993), designed to measure the five personality factors of the AFFM: Impulsive Sensation Seeking (ImpSS), Neuroticism–Anxiety (N–Anx), Aggression–Hostility (Agg–Host), Activity (Act), and Sociability (Sy).
Measures of Sensation Seeking
2015, Measures of Personality and Social Psychological ConstructsExamining bias in the impulsive sensation seeking (ImpSS) Scale using Differential Item Functioning (DIF) - An item response analysis
2011, Personality and Individual DifferencesCitation Excerpt :Collected responses were not associated with participants’ identifying information. These methods are supported by research examining the equivalence of paper and pencil versus Internet administration of personality measures, such as the ZKPQ (Zuckerman et al., 1993), which found no difference in factor structure or mean score between formats (Aluja, Rossier, & Zuckerman, 2007; Chuah, Drasgow, & Roberts, 2006; Gosling, Vazire, Srivastava, & John, 2004). The local institutional review board approved all study procedures.
Support for a two-factor model of impulsivity and hazardous substance use in British and Australian young adults
2011, Journal of Research in PersonalityCitation Excerpt :While the two testing sites differed in the method in which the questionnaires were administered, there is substantial evidence demonstrating little difference in pencil-and-paper versus internet administration of self-report instruments (Gosling, Vazire, Srivastava, & John, 2004). Studies have shown that personality measures retain their psychometric properties when administered online (Buchanan & Smith, 1999; Chuah, Drasgow, & Roberts, 2006), including measures of impulsivity (Aluja, Rossier, & Zuckerman, 2007). Comparative studies have also shown that alcohol and drug use assessments retain satisfactory psychometric properties when administered as an online questionnaire compared to an interview or pencil-and-paper questionnaire, even when participants do not complete the entire measure in one sitting (Brodey et al., 2004; Miller et al., 2002; Thomas & McCambridge, 2008).
Internet-Based Psychological Testing and Assessment
2011, Online Counseling, 2nd ed.