A Multitrait-Multimethod Validation of the Implicit Association Test
Abstract
Abstract. Recent theoretical and methodological innovations suggest a distinction between implicit and explicit evaluations. We applied Campbell and Fiske's (1959) classic multitrait-multimethod design precepts to test the construct validity of implicit attitudes as measured by the Implicit Association Test (IAT). Participants (N = 287) were measured on both self-report and IAT for up to seven attitude domains. Through a sequence of latent-variable structural models, systematic method variance was distinguished from attitude variance, and a correlated two-factors-per-attitude model (implicit and explicit factors) was superior to a single-factor-per-attitude specification. That is, despite sometimes strong relations between implicit and explicit attitude factors, collapsing their indicators into a single attitude factor resulted in relatively inferior model fit. We conclude that these implicit and explicit measures assess related but distinct attitude constructs. This provides a basis for, but does not distinguish between, dual-process and dual-representation theories that account for the distinctions between constructs.
References
Bosson J.K. , Swann W.B. , Pennebaker J.W. (2000). Stalking the perfect measure of self-esteem: The blind men and the elephant revisited?. Journal of Personality and Social Psychology, 79, 631– 643Browne M.W. (1991). [FITMOD: A computer program for calculating point and interval estimates of fit measures]. Unpublished manuscriptBrowne M.W. , Cudeck R. (1993). Alternative ways of assessing model fit. In K.A. Bollen & J.S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: SageCampbell D.T. , Fiske D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81– 105Chaiken S. , Trope Y. (1999). Dual-process theories in social psychology . New York: GuilfordConrey F.R. , Sherman J.W. , Gawronski B. , Hugenberg K. , Groom C. (2005). Beyond automaticity and control: The quad-model of behavioral response. Journal of Personality and Social Psychology, 89, 469– 487Cronbach L.J. , Meehl P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281– 302Cunningham W.A. , Johnson M.K. , Gatenby J.C. , Gore J.C. , Banaji M.R. (2003). Neural components of social evaluation. Journal of Personality and Social Psychology, 85, 639– 649Cunningham W.A. , Johnson M.K. , Raye C.L. , Gatenby J.C. , Gore J.C. , Banaji M.R. (2004). Separable neural components in the processing of Black and White faces. Psychological Science, 15, 806– 813Cunningham W.A. , Nezlek J.B. , Banaji M.R. (2004). Implicit and explicit ethnocentrism: Revisiting the ideologies of prejudice. Personality and Social Psychology Bulletin, 30, 1332– 1346Cunningham W.A. , Preacher K.J. , Banaji M.R. (2001). Implicit attitude measures: Consistency, stability, and convergent validity. Psychological Science, 12, 163– 170De Houwer J. (2001). A structural and process analysis of the Implicit Association Test. Journal of Experimental Social Psychology, 37, 443– 451Enders C.K. , Bandalos D.L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 8, 430– 457Fazio R.H. , Olson M.A. (2003). Implicit measures in social cognition research: Their meaning and use. Annual Review of Psychology, 54, 297– 327Fazio R.H. , Towles-Schwen T. (1999). The MODE model of attitude-behavior processes. In S. Chaiken & Y. Trope (Eds.), Dual process theories in social psychology (pp. 97-116). New York: GuilfordFazio R.H. , Sanbonmatsu D.M. , Powell M.C. , Kardes F.R. (1986). On the automatic activation of attitudes. Journal of Personality and Social Psychology, 50, 229– 238Gilbert D.T. (1999). What the mind's not. In S. Chaiken & Y. Trope (Eds.), Dual process theories in social psychology (pp. 3-11). New York: GuilfordGreenwald A.G. , Banaji M.R. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102(1), 4– 27Greenwald A.G. , Farnham S.D. (2000). Using the Implicit Association Test to measure self-esteem and self-concept. Journal of Personality and Social Psychology, 79, 1022– 1038Greenwald A.G. , McGhee D.E. , Schwartz J.L.K. (1998). Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74, 1464– 1480Greenwald A.G. , Nosek B.A. (2001). Health of the Implicit Association Test at age 3. Zeitschrift für Experimentelle Psychologie, 48, 85– 93Greenwald A.G. , Nosek B.A. , Banaji M.R. (2003). Understanding and using the Implicit Association Test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85, 197– 216Greenwald A.G. , Nosek B.A. in press Attitudinal dissociation: What does it mean?. In R.E. Petty, R.H. Fazio, & P. Briñol (Eds.), Attitudes: Insights from the new implicit measures. Hillsdale, NJ: ErlbaumHofmann W. , Gschwendner T. , Nosek B.A. , Schmitt M. (2005). What moderates implicit-explicit consistency?. European Review of Social Psychology, 16, 335– 390Jöreskog K.G. (1974). Analyzing psychological data by structural analysis of covariance matrices. In R.C. Atkinson, D.H. Krantz, R.D. Luce, & P. Suppes (Eds.), Contemporary developments in mathematical psychology: Measurement, psychophysics, and neural information processing (Vol. 2, pp. 1-56). San Francisco: FreemanJöreskog K.G. , Sörbom D. (1979). In J. Magidson (Ed.), Advances in factor analysis and structural equation models. Cambridge, MA: Abt BooksJost J.T. , Banaji M.R. , Nosek B.A. (2004). A decade of system justification theory: Accumulated evidence for the unconscious bolstering of the status quo. Political Psychology, 25, 881– 919Karpinski A. , Hilton J.L. (2001). Attitudes and the Implicit Association Test. Journal of Personality and Social Psychology, 81, 774– 788Loehlin J.C. (2004). Latent variable models: An introduction to factor, path, and structural analysis . Hillsdale, NJ:ErlbaumMacCallum R.C. , Browne M.W. , Sugawara H.M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130– 149Marsh H.W. , Grayson D. (1995). Latent variable models of multitrait-multimethod data. In R.H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 177-198). Thousand Oaks, CA: SageMcArdle J.J. (1994). Structural factor analysis experiments with incomplete data. Multivariate Behavioral Research, 29, 409– 454McArdle J.J. , Prescott C.A. (1992). Age-based construct validation using structural equation modeling. Experimental Aging Research, 18, 87– 115McDonald R.P. (1985). Factor analysis and related methods . Hillsdale, NJ: ErlbaumMierke J. , Klauer K.C. (2003). Method-specific variance in the Implicit Association Test. Journal of Personality and Social Psychology, 85, 1180– 1192Mitchell J.P. , Nosek B.A. , Banaji M.R. (2003). Contextual variations in implicit evaluation. Journal of Experimental Psychology: General, 132(3), 455– 469Muthén L.K. , Muthén B.O. (1998-2004). Mplus user's guide (3rd ed.). Los Angeles: Muthén & MuthénNosek B.A. (2005). Moderators of the relationship between implicit and explicit evaluation. Journal of Experimental Psychology: General, 135, 565– 584Nosek B.A. , Banaji M.R. (2001). The go/no-go association task. Social Cognition, 19, 625– 666Nosek B.A. , Greenwald A.G. , Banaji M.R. (2005). Understanding and using the Implicit Association Test: II. Method variables and construct validity. Personality and Social Psychology Bulletin, 31(2), 166– 180Nosek B.A. , Greenwald A.G. , Banaji M.R. (2006). The implicit association test at age 7: A methodological and conceptual review. In J.A. Bargh (Ed.), Automatic Processes in Social Thinking and Behavior (pp. 265-292). New York: Psychology PressPoehlman T.A. , Uhlmann E. , Greenwald A.G. , Banaji M.R. (2004). Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity . Unpublished manuscript, Yale University, New Haven, CT2002). A virtual laboratory for research in implicit social cognition. Version 1.5 . http://implicit.harvard.edu/
(Rothermund K. , Wentura D. (2004). Underlying processes in the Implicit Association Test (IAT): Dissociating salience from associations. Journal of Experimental Psychology: General, 133, 139– 165Steiger J.H. (2000). Point estimation, hypothesis testing, and interval estimation using the RMSEA: Some comments and a reply to Hayduk and Glaser. Structural Equation Modeling, 7, 149– 162Steiger J.H. , Lind J.C. (1980, May). Statistically based tests for the number of factors . Paper presented at the annual spring meeting of the Psychometric Society, Iowa City, IAStrack F. , Deutsch R. (2004). Reflective and impulsive determinants of social behavior. Personality and Social Psychology Review, 3, 220– 247Widaman K.F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1– 26Wilson T.D. , Lindsey S. , Schooler T.Y. (2000). A model of dual attitudes. Psychological Review, 107(1), 101– 126Wittenbrink B. , Judd C.M. , Park B. (1997). Evidence for racial prejudice at the implicit level and its relationship with questionnaire measures. Journal of Personality and Social Psychology, 72, 262– 274