Does online psychological test administration facilitate faking?

Abstract

This study examined for the first time the effect of delivery mode on faking good and faking bad in psychological testing. Participants (N = 223) completed questionnaires either online or in pen-and-paper format in a mixed experimental design. After completing measures of personality (HEXACO-60, Ashton & Lee, 2009) and depression (DASS-21, Lovibond & Lovibond, 1995) under standard instructions, participants then faked the personality measure as if applying for a job, and faked the depression measure as if experiencing severe depression. Equivalence of internet and pen-and paper-administration on faking was then measured between groups. As predicted, participants were able to fake good on the HEXACO-60 and to fake bad on the DASS-21. Also as predicted, there were no significant differences in faked scores as a function of test administration mode. Further, examination of effect sizes confirmed that the influence of test administration mode was small. It was concluded that online and pen-and paper presentation are largely equivalent when an individual is faking responses in psychological testing. Given the advantages of online assessment and the importance of valid psychological testing, future research should investigate whether the current findings can be generalised to other faking and malingering scenarios and to other psychological measures.

Introduction

The internet is being increasingly used for psychological research and assessment in a number of contexts, including for vocational (Piotrowski & Armstrong, 2006) and clinical (Hedman et al., 2010) purposes. Much research has examined the equivalence of pen-and-paper and web-based versions of specific psychological measures in a variety of domains (e.g. Coles et al., 2007, Denniston et al., 2010, Hedman et al., 2010, Lewis et al., 2009; Templer & Lange, 2008). However, the equivalence of psychological test presentation via the internet and pen-and-paper has not been examined in regards to the susceptibility of self-reports to faking. This study aimed to explore for the first time the influence that mode of delivery elicits on the fakability of self-report psychological tests.

The internet has rapidly become a valuable medium for data collection as it is considered inexpensive, easily accessible, and discrete (Birnbaum, 2004). Other advantages of on-line data collection are that participants can be required to endorse answers to all items (thereby minimising missing data), and that data can be transferred electronically for analysis (thereby reducing data entry error) (Carlbring et al., 2007; Lewis et al., 2009). However, rather than assuming that internet and pen-and-paper administrations of psychological measures are interchangeable, it has been recommended that all psychological measures be evaluated to investigate whether internet and pen-and-paper administrations are comparable (Buchanan, 2002).

To date, a number of tests have been compared, including clinical measures (e.g. Carlbring et al., 2007, Coles et al., 2007, Herrero and Meneses, 2006), personality measures (e.g. Templer & Lange, 2008), ability measures (e.g. Ihme et al., 2009), and health- and risk-related behaviour measures (e.g. Horswill and Coster, 2001, Lewis et al., 2009, McCabe et al., 2006, Whittier et al., 2004). Overall, findings suggest that the internet is both a feasible and largely comparable method for conducting psychological testing. However, no extant research has examined the role of mode of delivery in the administration of self-report psychometric tests and the potential facilitation of faking behaviour.

Faking or malingering occurs when an individual strategically alters their self representation in a particular test (Grieve & Mahar, 2010). Faking good is characterised by responses that augment an individual’s actual state, making them appear psychologically superior (for example, in a job application), while faking bad occurs when an individual presents themselves as psychologically worse than they actually are (for example, to be diagnosed with a disorder).

Faking of psychological assessments may have a number of consequences. For example, in vocational contexts, faking will not only influence who gets hired (Mueller-Hanson, Heggestad, & Thornton, 2003), but can also impact the subsequent training and management of employees (Landers, Sackett, & Tuzinski, 2011). In clinical contexts, faking may influence access to therapy or medication (Suhr, Hammers, Dobbins-Buckland, Zimak, & Hughes, 2008).

This study aimed to build on the existing research regarding the validity of pen-and-paper and online testing methods by investigating whether administration mode influences an individual’s ability to fake a measure. To more fully address this aim, both faking good and faking bad scenarios were employed.

Previous research has shown that individuals are readily able to fake good in vocational contexts (for example, as if applying for a job) by maximising positive, job-relevant personality aspects and minimising negative personality aspects (Mahar et al., 2006). Therefore, an initial hypothesis was that participants would be able to alter their original personality profiles to a more positive faked profile when asked to complete a personality measure as if they were applying for a job. Specifically, it was anticipated that the faked profiles would score significantly higher than the original profiles on desirable employee characteristics (honesty/humility, extraversion, agreeableness, conscientiousness, and openness), and significantly lower on undesirable employee characteristics (emotionality).

The second hypothesis addressed the main research question. Given that most research into the equivalence of online and pen-and-paper personality testing has found that both modes of administration elicit similar test results (e.g. Templer & Lange, 2008), it was hypothesised that faked profile scores would be equivalent regardless of which mode of administration was used. While it is acknowledged that this is in fact testing the null hypothesis, and that it is difficult to ascertain whether a hypothesis of no difference is true (Nickerson, 2000), a hypothesis of this nature was required by the research question. It follows that, in order to test the second hypothesis, a close examination of effect size, rather than statistically significant differences alone, was indicated.

It has also been shown that individuals are able to fake bad in clinical contexts (for example, as if they have depression, see Grieve & Mahar, 2010). Therefore, it was hypothesised that when participants were asked to complete a depression measure as if they had depression, they would be able to alter their original scores on that measure to faked scores suggesting a provisional diagnosis of depression.

In order to address the main research question, scores were compared between groups of participants who faked the depression measure either online or using pen-and-paper. Again, as previous research has largely supported the equivalence of the two modes of administration for clinical measures (e.g. Carlbring et al., 2007), it was hypothesised that there would be no significant differences in faked depression scores as a function of administration method. Once more, as this prediction was testing the null hypothesis, close examination of the effect size was also undertaken.