Malleability of attitudes or malleability of the IAT?

https://doi.org/10.1016/j.jesp.2009.11.011Get rights and content

Abstract

In the current set of experiments, we establish, and explore the consequences of, the imprecision that characterizes the attribute response labels typically employed in the Implicit Association Test (IAT). In Experiment 1, we demonstrate the malleability of the IAT, as conventionally implemented. IAT scores are shown to be influenced by perspective mindsets induced by an unrelated preceding task. Then, we explore how the malleability of the IAT can lead to the inference that attitude change has occurred even when there is very good reason to believe it has not (Experiment 2), and conversely, how it can obscure the detection of attitude change when such change is indeed likely to have occurred (Experiment 3). We provide conceptual explanations for these discrepancies and suggest methodological improvements to enhance the specificity of IAT measures.

Introduction

Most behavioral scientists who employ questionnaire measures have experienced the occasional, troubling realization that participants interpreted a specific question differently than they had intended. Whether the cause was poor wording, the implications of a preceding set of questions, or some unexpected natural event that cast a different light on the matter at hand, it became apparent that the participants’ responses were less than correspondent to the query we had been posing. Indeed, years of research have been devoted to the study of survey responding as an exercise in communication between the questioner and the respondent; “questions shape answers” (Schwarz, 1999, p. 93). It has been shown repeatedly that when survey questions are ambiguous, participants guess or use contextual information to disambiguate and respond as best they can (Bickart, 1992, Billiet et al., 1992, Krosnick, 1992, Krosnick and Alwin, 1987). Although they may stem from a desire to be cooperative, the outcome of such efforts to disambiguate have the potential to produce less than meaningful data and can lead the researcher to draw inappropriate inferences. Hence, experienced survey researchers consistently strive to construct questions that are free of ambiguities.

Perhaps because of the basic assumptions underlying implicit measures (e.g., Greenwald & Banaji, 1995), the extent to which these attitude measurement tools are influenced by efforts on the part of the participants to disambiguate the nature of the task before them has not yet been as carefully examined. Researchers have devoted considerable effort to determining exactly what measures like the Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998) are assessing, as well as the mechanisms that underlie the measurement procedures (e.g., De Houwer, 2009, Fazio and Olson, 2003, Govan and Williams, 2004, Karpinski and Hilton, 2001). However, more research is needed so as to inform the scientific community about the appropriateness of inferences that might be drawn from IAT findings. Most importantly, the implications of any disambiguating efforts by the participants for inferences regarding the measurement procedure vs. underlying representation require careful analysis. When change is observed on the IAT, does that change reflect the malleability of attitudes or the malleability of the measure? We argue that the imprecise response labels typically employed in the IAT allow for plasticity that can lead to erroneous inferences regarding the malleability of attitudes.

Specifically, the present research demonstrates that the IAT as conventionally implemented is open to multiple interpretations, and therefore, can provide contextually malleable measurement outcomes (Experiment 1). We then explore important consequences of the IAT’s malleability for the inferences that can be drawn regarding the presence or absence of attitude change following a social influence attempt. Whereas past research often has interpreted change in IAT scores as evidence of the malleability of attitudes, we argue that the malleability of the IAT can sometimes lead to the mistaken inference that attitude change has occurred even when there is very good reason to believe it has not (Experiments 2). However, we also show that the traditional implementation of the IAT can sometimes obscure the detection of attitude change when such change is indeed likely to have occurred (Experiment 3). We provide a conceptual explanation for these discrepancies and suggest methodological improvements to focus IAT measures more precisely.

Since its inception, the IAT has become the preferred implicit measure for many psychological variables. Because implicit measures are presumed to be relatively immune from many of the concerns that plague self-report measures, the IAT has been especially useful in domains in which social desirability is a concern. This has led to its extensive use in areas such as stereotyping and prejudice (e.g., Blair, 2002, Greenwald et al., 1998, Nosek et al., 2005), health behaviors (e.g., Sherman, Presson, Chassin, Rose, & Koch, 2002), and self-esteem (e.g., Greenwald and Farnham, 2000, Jordan et al., 2003). Despite this wide usage, evidence indicates that the IAT as typically implemented may not be as robust to momentary, irrelevant contextual considerations as once believed (e.g., Karpinski and Hilton, 2001, Olson and Fazio, 2004).

IAT performance depends on one’s ability to cope with a response mapping problem. During the critical blocks, any given response key has two meanings: one key is to be pressed, for example, whenever an exemplar of Category X or something “good” is presented, and the other key whenever an exemplar of Category Y or something “bad” is presented. To perform well, one needs to remember the dual meaning of any given key. The easier it is to associate X with good and/or Y with bad, the faster one can respond. The assumption is that the ease of remembering and working with a given response mapping is a function of individuals’ attitudes toward X and Y. But, why would attitudes be the only form of information to affect the efficiency with which the dual meaning of a response key can be accommodated?

Olson and Fazio (2004) argued that IAT participants may be influenced by any information that can facilitate their management of the response mapping, including information that is inconsistent with their attitudes. They maintained that category labels such as the commonly employed “good/bad” or “pleasant/unpleasant” are open to multiple interpretations and, hence, allow for the activation of any information that might be useful for managing the demands of the categorization task. Most importantly, the perspective intended by such category labels is unspecified: “good”/“bad” or “pleasant”/“unpleasant” to whom? Should the stimuli be categorized from the perspective of the self, the culture, or the researcher, to consider just a few possibilities?

An important consequence of this category label ambiguity is that the attitude estimates provided by traditional versions of the IAT can be influenced by extrapersonal associations (Han et al., 2006a, Olson and Fazio, 2004) – attitude-irrelevant knowledge that does not form the basis of the individual’s attitude toward to the object. Such extrapersonal knowledge sometimes arises from the recognition that others have attitudes that differ from one’s own, whether those others be a specific individual, a class of people, or even more general cultural norms. For example, the knowledge that my neighbor was a huge supporter of presidential candidate McCain did not impact my support for Obama. Nevertheless, such knowledge can facilitate a respondent’s efforts to accommodate the dual meaning of the response keys during the IAT. Thinking about Aunt Mary or my neighbor’s preferences, the researcher’s presumed intent, or the cultural perspective may promote faster responding for a given response mapping.

A much more thorough and lengthy consideration of extrapersonal associations can be found in Olson, Fazio, and Han (2009). Responding to various questions about how “extrapersonal” might be conceptualized (Gawronski et al., 2008, Nosek and Hansen, 2008a), Olson, Crawford, and Devlin (2009) address issues concerning the conceptual and empirical distinction between personal and extrapersonal associations, including definitional matters, the development of extrapersonal knowledge, and the ensuing advantage that typically characterizes personal associations in terms of their likelihood of activation. In the present context, we wish to emphasize only three related points. The first is definitional, and concerns both attitudes and extrapersonal associations. Consistent with such classic definitions as those of Allport (1935) and Thurstone (1928), we view attitudes as inherently personal reactions to an attitude object. More specifically, we consider attitudes to be summary evaluations that have the potential to be automatically activated upon encountering an attitude object and that then guide construal of the object in the immediate situation and ultimately approach/avoidance decisions (see Fazio, 2007). Simply put, extrapersonal associations are items of information that, although associated with the attitude object, have not contributed to one’s summary evaluation of that object. This may be because the information was deemed inconsequential or irrelevant to one’s personal tastes, or because it was rejected as untrue (Petty, Briñol, & DeMarree, 2007). Or, it may simply be because the information played no role as the attitude developed. An individual with a severe allergy to peanuts in all likelihood developed a negative attitude as a consequence of the first causally identifiable allergic reaction, and this aversion will have been reinforced by the individual’s regular surveillance of food items and menus for the presence of peanuts. A sibling’s liking for peanut butter played no role in the development of this negative attitude, nor did other knowledge related to peanuts, such as the cuteness of the Mr. Peanut character (a peanut sporting a top hat and cane) that serves as the mascot for Planters Peanuts. Information of this sort is certainly available in memory, but it did not contribute to the person’s negative attitude toward peanuts.

The second point to be highlighted follows directly from this conceptualization. Extrapersonal associations should not be regarded as corresponding directly to cultural associations (Nosek & Hansen, 2008a). There is no necessary equivalence between the two. Indeed, cultural knowledge, i.e., how the attitude object is generally portrayed within the culture, may serve as the very basis for a given individual’s attitude. It is when personal attitudes deviate from the more cultural, normative view that such cultural knowledge can be considered extrapersonal for a person. Most lovers of anchovies recognize that their positivity places them in the minority, but this does not stop them from anticipating delight at the prospect of requesting that their caesar salads be served with anchovies. For such individuals, knowledge of the more culturally predominant negativity assumes the status of an extrapersonal association.

Finally, we wish to emphasize that our focus on the concept of extrapersonal associations is not intended to deny in any way the essential social psychological principle that social influence is pervasive in nature. Individuals develop attitudes not only from their personal experiences when interacting with the attitude object, but also through the social transmission of information about the attitude object, the norms that characterize their reference groups, and even their mere awareness of others’ attitudes. Social forces have long been known to produce not only public compliance, but also private acceptance, at least under certain specifiable conditions (e.g., Deutsch & Gerard, 1955). Informational social influence is undoubtedly a powerful force, as is direct persuasion. Yet, it also seems evident that at least sometimes the attitudes of at least some people are not affected by their knowledge of others’ views or the communicated attributes of the attitude object. Social influence phenomena are not universal. It is when they fail to impact a given individual’s attitudes that such information meets our definition of an extrapersonal association. To borrow an example from Olson et al. (2009), awareness of one’s mother’s obvious affection for floral-patterned wallpaper, bedspreads, and curtains represents an extrapersonal association to the extent that one’s own reaction upon seeing a floral pattern is a sense of distaste. Even if the distaste had been facilitated by a desire to be different from mother, knowledge of her positivity is distinct from one’s experienced negativity.

Given the idiosyncratic nature of attitudes and the varying bases from which individuals’ attitudes might develop, it is difficult to identify a given item of information, in any a priori manner, as personal or extrapersonal for any given individual (see Olson et al., 2009, for further discussion of this issue). What is extrapersonal to one person may form the very essence of another’s attitude. Hence, the most convincing evidence that extrapersonal associations can indeed influence performance on the IAT stems from research that has experimentally created both attitudes towards novel stimuli and extrapersonal associations in the laboratory. With such experimental control, Han et al. (2006a) were able to engineer a situation in which it was highly implausible that a particular piece of information would impact individuals’ attitudes. Hence, the information could confidently be considered extrapersonal in nature.

In these experiments, participants first formed attitudes towards two novel Pokémon characters, which were characterized by attributes that made one objectively superior to the other, and hence, clearly preferred. A questionnaire administered at this point in the procedure induced participants to rehearse and express their attitudes multiple times, thus establishing a very firm preference. Participants then were exposed to a video of two 10-years-old boys commenting on the same characters. Depending on the condition, the boys either agreed with participants as to which character was superior (consistent condition) or disagreed, expressing an unjustified, clearly erroneous, and easily dismissed preference for the objectively inferior character (inconsistent or extrapersonal condition).

Across two experiments, results revealed that participants’ IAT scores differed as a function of the comments to which they had been exposed. When exposed to inconsistent commentary (i.e., when the boys’ opinions contrasted with objective reality and participants’ own attitudes), participants’ IAT scores suggested a significantly reduced preference for the objectively superior Pokémon character compared to when the boys agreed. This change in IAT scores occurred despite the fact that participants in this extrapersonal condition both rated the boys as irrational and foolish and chose the superior Pokémon card over the other as a parting bonus at the end of the study. Moreover, in contrast to the traditional IAT, a subliminal priming measure of attitudes (in Experiment 1; Han et al., 2006a), and a personalized version of the IAT (in Experiment 2), where the “pleasant/unpleasant” labels were changed to “I like/I don’t like,” remained unaffected by the introduction of the inconsistent video. Both measures revealed only a preference for the objectively superior Pokémon. Apparently, even though their own preferences were unaffected, participants in the extrapersonal condition were able to recruit the boys’ opinion to solve the mapping problem posed by assignment of the inferior Pokémon character to the same key as “pleasant” and the superior Pokémon to the same key as “unpleasant.” By momentarily adopting the boys’ perspective, it proved easier to associate the inferior Pokémon with the key signifying positivity.

Section snippets

Goals and overview of studies

Since we argue that the valence labels of the traditional IAT are potentially ambiguous, we first sought to demonstrate the openness of these labels to multiple interpretations and the resulting malleability of the traditional IAT. In Experiment 1, we explored how an unrelated preceding task can change participants’ perspective regarding the traditional IAT category labels. Our findings suggest that, depending on the perspective primed by the preceding task (self or other people), participants’

Experiment 1

It is well known that previous tasks can influence how people interpret and answer subsequent questions. For example, after being asked to select two high quality TV shows from a list of 10, participants rated TV programming as generally more satisfying than after selecting two low quality TV shows (Bless & Wänke, 2000). Similarly, Salancik and Conway (1975) demonstrated that having participants complete a behavioral inventory that pairs pro-religious behaviors with the adverb “occasionally”

Experiment 2

In Experiment 2, we explored whether the manipulations employed by Foroni and Mayr (2005), instead of influencing attitudes, may have provided participants with highly salient extrapersonal associations that then could be used to manage the response mapping problem posed by the IAT. As noted earlier, we do not find it very plausible that a brief imaginary excursion into a counterfactual world could produce a change in longstanding attitudinal representations regarding insects and flowers. To

Experiment 3

Our previous findings show that the malleability of label construal leaves the traditional IAT susceptible to the influence of extrapersonal associations. As a result, the measure can suggest attitude change when none has occurred. What happens if, following a persuasion attempt, attitude change does occur? Our last experiment illustrates the reverse side of the IAT’s malleability, namely that the IAT as conventionally implemented can sometimes fail to reveal attitude change when such change is

General discussion

The goal of the present experiments was to explore the malleability that results from the multiple plausible meanings permitted by the evaluative labels of the traditional IAT. We also investigated the consequences of the IAT’s malleability by reexamining prior research interpreting that malleability as an evidence of attitude change when no such change seemed likely. In addition, we showed that the malleability of the IAT can sometimes obscure the detection of attitude change when the

Conclusion

In conclusion, the IAT’s sensitivity to extrapersonal associations leaves it susceptible to momentary contextual influences – ones for which it is difficult to plausibly entertain that attitudes have changed. Presumably, no researcher would want their measure of attitudes to be influenced by the perspective mindset induced by a preceding task, or by a brief mental excursion into an obviously fictional future world. We doubt anyone would argue that these preceding events changed attitudes. Yet,

Acknowledgments

Portions of this research were presented at the annual meeting of the Society for Personality and Social Psychology Albuquerque, February 2008, and at the annual meeting of the Midwestern Psychological Association, Chicago, May 2008. This research was supported by Grant MH38832 from the National Institute of Mental Health and by an A Grant from the HEC Paris Foundation. We thank Geoff Kaufman and Chris Loersch for their thoughtful comments on earlier version of this manuscript, and Tim Sbory,

References (57)

  • H. Bless et al.

    Can the same information be typical and atypical? How perceived typicality moderates assimilation and contrast in evaluative judgments

    Personality and Social Psychology Bulletin

    (2000)
  • P. Briñol et al.

    Changing attitudes on implicit versus explicit measures: What is the difference?

  • Carpenter, S., Banaji, M. R. (2001). Malleability in implicit stereotype and attitudes. Paper presented at the annual...
  • M.T. Conner et al.

    Relations between implicit and explicit measures of attitudes and measures of behavior: Evidence of moderation by individual difference variables

    Personality and Social Psychology Bulletin

    (2007)
  • N. Dasgupta et al.

    On the malleability of automatic attitudes: Combating automatic prejudice with images of admired and disliked individuals

    Journal of Personality and Social Psychology

    (2001)
  • J. De Houwer

    Comparing measures of attitudes at the functional and procedural level: Analysis and implications

  • J. De Houwer et al.

    Do smokers have a negative implicit attitude toward smoking?

    Cognition & Emotion

    (2006)
  • J. De Houwer et al.

    How to define and examine the implicitness of implicit measures

  • M. Deutsch et al.

    A study of normative and informational social influences upon individual judgment

    Journal of Abnormal and Social Psychology

    (1955)
  • R.H. Fazio

    Attitudes as object-evaluation associations: Determinants, consequences, and correlates of attitude accessibility

  • R.H. Fazio

    Attitudes as object-evaluation associations of varying strength

    Social Cognition

    (2007)
  • R.H. Fazio et al.

    Implicit measures in social cognition research: Their meaning and use

    Annual Review of Psychology

    (2003)
  • F. Foroni et al.

    The power of a story: New, automatic associations from a single reading of a short scenario

    Psychonomic Bulletin & Review

    (2005)
  • B. Gawronski et al.

    What makes mental associations personal or extra-personal? Conceptual issues in the methodological debate about implicit attitude measures

    Social and Personality Psychology Compass

    (2008)
  • P.A. Goff et al.

    Not yet human: Implicit knowledge, historical dehumanization, and contemporary consequences

    Journal of Personality and Social Psychology

    (2008)
  • A.G. Greenwald et al.

    Implicit social cognition: Attitudes, self-esteem, and stereotypes

    Psychological Review

    (1995)
  • A.G. Greenwald et al.

    Using the Implicit Association Test to measure self-esteem and self-concept

    Journal of Personality and Social Psychology

    (2000)
  • A.G. Greenwald et al.

    Measuring individual differences in implicit cognition: The Implicit Association Test

    Journal of Personality and Social Psychology

    (1998)
  • Cited by (51)

    • Social norm coordination and readiness to change female genital cutting: Evidence from Senegambia

      2020, SSM - Population Health
      Citation Excerpt :

      Because IAT require a series of rapid judgments, they are thought to capture cognitive processes of which respondents do not have conscious awareness, and so to be free from the types of response bias that can influence explicit, survey-based measures (e.g., lack of awareness, reactivity, or social desirability bias). However, IATs’ freedom from bias has become the subject of debate as research finds that implicit attitudes may be influenced by the context of the test administration (Fazio & Olson, 2003, 2014; Han, Czellar, Olson, & Fazio, 2010). Further, in the context of FGC and the social coordination model, IAT may not capture the complexity of attitudes and decision-making, including awareness of both costs and benefits to cutting.

    View all citing articles on Scopus
    View full text