Does online psychological test administration facilitate faking?
Highlights
► We examined whether online or traditional test administration influences fakability. ► Administration mode did not influence scores when faking good. ► Administration mode did not influence scores when faking bad. ► Online and pen-and paper presentation appear equivalent when an individual is faking. ► Future research should investigate other measures and faking scenarios.
Introduction
The internet is being increasingly used for psychological research and assessment in a number of contexts, including for vocational (Piotrowski & Armstrong, 2006) and clinical (Hedman et al., 2010) purposes. Much research has examined the equivalence of pen-and-paper and web-based versions of specific psychological measures in a variety of domains (e.g. Coles et al., 2007, Denniston et al., 2010, Hedman et al., 2010, Lewis et al., 2009; Templer & Lange, 2008). However, the equivalence of psychological test presentation via the internet and pen-and-paper has not been examined in regards to the susceptibility of self-reports to faking. This study aimed to explore for the first time the influence that mode of delivery elicits on the fakability of self-report psychological tests.
The internet has rapidly become a valuable medium for data collection as it is considered inexpensive, easily accessible, and discrete (Birnbaum, 2004). Other advantages of on-line data collection are that participants can be required to endorse answers to all items (thereby minimising missing data), and that data can be transferred electronically for analysis (thereby reducing data entry error) (Carlbring et al., 2007; Lewis et al., 2009). However, rather than assuming that internet and pen-and-paper administrations of psychological measures are interchangeable, it has been recommended that all psychological measures be evaluated to investigate whether internet and pen-and-paper administrations are comparable (Buchanan, 2002).
To date, a number of tests have been compared, including clinical measures (e.g. Carlbring et al., 2007, Coles et al., 2007, Herrero and Meneses, 2006), personality measures (e.g. Templer & Lange, 2008), ability measures (e.g. Ihme et al., 2009), and health- and risk-related behaviour measures (e.g. Horswill and Coster, 2001, Lewis et al., 2009, McCabe et al., 2006, Whittier et al., 2004). Overall, findings suggest that the internet is both a feasible and largely comparable method for conducting psychological testing. However, no extant research has examined the role of mode of delivery in the administration of self-report psychometric tests and the potential facilitation of faking behaviour.
Faking or malingering occurs when an individual strategically alters their self representation in a particular test (Grieve & Mahar, 2010). Faking good is characterised by responses that augment an individual’s actual state, making them appear psychologically superior (for example, in a job application), while faking bad occurs when an individual presents themselves as psychologically worse than they actually are (for example, to be diagnosed with a disorder).
Faking of psychological assessments may have a number of consequences. For example, in vocational contexts, faking will not only influence who gets hired (Mueller-Hanson, Heggestad, & Thornton, 2003), but can also impact the subsequent training and management of employees (Landers, Sackett, & Tuzinski, 2011). In clinical contexts, faking may influence access to therapy or medication (Suhr, Hammers, Dobbins-Buckland, Zimak, & Hughes, 2008).
This study aimed to build on the existing research regarding the validity of pen-and-paper and online testing methods by investigating whether administration mode influences an individual’s ability to fake a measure. To more fully address this aim, both faking good and faking bad scenarios were employed.
Previous research has shown that individuals are readily able to fake good in vocational contexts (for example, as if applying for a job) by maximising positive, job-relevant personality aspects and minimising negative personality aspects (Mahar et al., 2006). Therefore, an initial hypothesis was that participants would be able to alter their original personality profiles to a more positive faked profile when asked to complete a personality measure as if they were applying for a job. Specifically, it was anticipated that the faked profiles would score significantly higher than the original profiles on desirable employee characteristics (honesty/humility, extraversion, agreeableness, conscientiousness, and openness), and significantly lower on undesirable employee characteristics (emotionality).
The second hypothesis addressed the main research question. Given that most research into the equivalence of online and pen-and-paper personality testing has found that both modes of administration elicit similar test results (e.g. Templer & Lange, 2008), it was hypothesised that faked profile scores would be equivalent regardless of which mode of administration was used. While it is acknowledged that this is in fact testing the null hypothesis, and that it is difficult to ascertain whether a hypothesis of no difference is true (Nickerson, 2000), a hypothesis of this nature was required by the research question. It follows that, in order to test the second hypothesis, a close examination of effect size, rather than statistically significant differences alone, was indicated.
It has also been shown that individuals are able to fake bad in clinical contexts (for example, as if they have depression, see Grieve & Mahar, 2010). Therefore, it was hypothesised that when participants were asked to complete a depression measure as if they had depression, they would be able to alter their original scores on that measure to faked scores suggesting a provisional diagnosis of depression.
In order to address the main research question, scores were compared between groups of participants who faked the depression measure either online or using pen-and-paper. Again, as previous research has largely supported the equivalence of the two modes of administration for clinical measures (e.g. Carlbring et al., 2007), it was hypothesised that there would be no significant differences in faked depression scores as a function of administration method. Once more, as this prediction was testing the null hypothesis, close examination of the effect size was also undertaken.
Section snippets
Participants
The sample consisted of 223 participants (54 men, 169 women) who completed the questionnaire on the internet (63%) or on paper (37%). Participants were recruited from the student body of an Australian university (41.5%), and the general public (51.8%). 6.7% of participants did not report whether or not they were students. Participants were invited to participate via in-class announcements, word of mouth, and using the social networking website Facebook. Participation was voluntary and no
Manipulation check
Answers to the manipulation check were reviewed to ensure that participants had understood and followed the experimental manipulation, and were dummy coded as either ‘followed instructions’ or ‘did not follow instructions’. Examples of responses coded as successfully facilitated were “to bend the truth or lie straight out to get the job” and “trying to answer like a depressed person would”. Examples of responses coded as not following instructions included “to answer honestly and openly” and
Discussion
The aim of this study was to examine whether internet and pen-and-paper test administrations are equally susceptible to faking for measures of personality and depression. As predicted, participants were able to alter their test profiles to present themselves as a desirable employee, and also to appear as if a provisional diagnosis of depression was indicated. When faking good, emotionality tended to be under-reported, while honesty/humility, extraversion, agreeableness, conscientiousness and
Conclusions
This research examined for the first time the fakability of two measures (the HEXACO-60 and the DASS-21) as a function of test administration mode. The small effect sizes indicated that internet administration and pen-and-paper administration are largely equivalent when an individual is engaging in faking behaviours on these specific self-report measures of personality (faking good) and depression (faking bad). While future research should extend investigation to other contexts and measures,
Acknowledgements
The authors would like to thank Catherine McSwiggan for her assistance in data collection. We would also like to thank two anonymous reviewers for their useful comments.
References (28)
- et al.
Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research
Computers in Human Behavior
(2007) - et al.
Assessing obsessive compulsive symptoms and cognitions on the internet: Evidence for the compatibility of paper and internet administration
Behavior Research and Therapy
(2007) - et al.
Comparison of paper-and-pencil versus Web administration of the Youth Risk Behavior Survey (YRBS): Participation, data quality, and perceived privacy and anonymity
Computers in Human Behavior
(2010) - et al.
Internet adminstration of self-report measures commonly used in research on social anxiety disorder: A psychometric evaluation
Computers in Human Behavior
(2010) - et al.
Short Web-based versions of the perceived stress (PSS) and Center for Epidemiological Studies-Depression (CESD) Scales: A comparison to pencil and paper responses among Internet users
Computers in Human Behavior
(2006) - et al.
The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories
Behavior Research and Therapy
(1995) - et al.
Stereotyping as a response strategy when faking personality questionnaires
Personality and Individual Differences
(2006) - et al.
Comparison of web and mail surveys for studying secondary consequences associated with substance use: Evidence for minimal mode effects
Addictive Behaviors
(2006) - et al.
The relationship of malingered test failure to self-reported symptoms and neuropsychological findings in adults referred for ADHD evaluation
Archives of Clinical Neuropsychology
(2008) - et al.
Internet testing: Equivalence between proctored lab and unproctored field conditions
Computers in Human Behavior
(2008)
The HEXACO-60: A short measure of the major dimensions of personality
Journal of Personality Assessment
Human research and data collection via the Internet
Annual Review of Psychology
Statistical treatment of the Solomon Four Group Design: A meta-analytic approach
Psychological Bulletin
Online assessment: Desirable or dangerous?
Professional Psychology: Research and Practice
Cited by (26)
HEXACO personality predicts counterproductive work behavior and organizational citizenship behavior in low-stakes and job applicant contexts
2018, Journal of Research in PersonalityCitation Excerpt :A meta-analysis of instructed faking studies found average changes of around three-quarters of a standard deviation (Ones, Viswesvaran, & Schmidt, 1993). In addition to studies of instructed faking on HEXACO personality in the laboratory environment (Grieve & De Groot, 2011; MacCann, 2013), a recent study by Anglim, Morse, De Vries, MacCann, and Marty (2017) compared large samples of job applicants and non-applicants on the HEXACO-PI-R and found that job applicants scored higher on honesty-humility, extraversion, agreeableness, and conscientiousness. Although there is relative consensus that response distortion occurs in applicant settings, there is less agreement about whether such settings also reduce the predictive validity of personality.
Employment testing online, offline, and over the phone: Implications for e-assessment
2016, Revista de Psicologia del Trabajo y de las OrganizacionesCitation Excerpt :Previous research does indicate that when faking, individuals form their own concept or schema of the desirable profile (Jansen, König, Kleinmann, & Melchers, 2012). The current study built on previous research (Grieve & de Groot, 2011) examining the equivalence of electronic assessment methods in a vocational context by including telephone administration and a specific applicant profile. Overall, the results support previous research indicating equivalence between online and pen-and-paper test administration (e.g., Bates & Cox, 2008; Carlbring et al., 2007; Casler et al., 2013; Williams & McCord, 2006), and between online, pen-and-paper, and telephone administration (Knapp & Kirk, 2003).
Response Bias, Malingering, and Impression Management
2015, Measures of Personality and Social Psychological ConstructsPredicting intentions to fake in psychological testing: Which normative beliefs are important?
2014, Revista de Psicologia del Trabajo y de las OrganizacionesMore of a (wo)man offline? Gender roles measured in online and offline environments
2013, Personality and Individual DifferencesCitation Excerpt :Researchers have generally found that online versions of traditional pen and paper tests are equivalent (Naus et al., 2009). Online and offline equivalence has been examined and established in a range of assessments, for example personality (Chuah et al., 2006; Grieve & de Groot, 2011; Salgado & Moscoso, 2003), clinical (Coles, Cook, & Blake, 2007; Grieve & de Groot, 2011; Naus et al., 2009) and intelligence (Franzis & Helge, 2003) measures. However, although many measures have been deemed equivalent, Buchanan (2002) stated that the equivalence of online and offline tests cannot just be assumed, and recommended that equivalence should be demonstrated for every test.
Construct validity of a personality assessment game in a simulated selection situation and the moderating roles of the ability to identify criteria and dispositional insight
2023, International Journal of Selection and Assessment