Abstract
There is growing interest in the study of individual differences in face recognition, including one of its hallmarks, holistic processing, which can be defined as a failure of selective attention to parts. These efforts demand that researchers be aware of, and try to maximize, the reliability of their measurements. Here we report on the reliability of measurements using the composite task (complete design), a measure of holistic processing that has been shown to have relatively good validity. Several studies have used the composite task to investigate individual differences, yet only one study has discussed its reliability. We investigate the reliability of composite-task measurements in eight data sets from five different samples of subjects. In general, we found reliability to be fairly low, but there was substantial variability across experiments. Researchers should keep in mind that reliability is a property of measurements, not of a task, and think about the ways in which measurements in a particular task may be improved before embarking on individual differences research.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A version of the composite task featuring half of the trials has been referred to as the “partial design” in the literature (Gauthier & Bukach, 2007); however, the partial design confounds response and congruency (all same trials are incongruent and all different trials are congruent), which has been found to be problematic, because congruency produces a response bias that cannot be separated from sensitivity without the other half of the trials (see Cheung, Richler, Palmeri, & Gauthier, 2008; Richler, Cheung, & Gauthier, 2011a; Richler & Gauthier, in press).
Note that this reliability was computed for 144 trials, and we report reliabilities for 160 trials in the present work.
Note that DeGutis et al.’s (2013) reproduction of this formula included a minor error in which the primary and control conditions were switched.
References
Busigny, T., Joubert, S., Felician, O., Ceccaldi, M., & Rossion, B. (2010). Holistic perception of the individual face is specific and necessary: Evidence from an extensive case study of acquired prosopagnosia. Neuropsychologia, 48, 4057–4092. doi:10.1016/j.neuropsychologia.2010.09.017
Callender, J. C., & Osburn, H. G. (1979). An empirical comparison of coefficient alpha, Guttman’s Lambda-2, and MSPLIT maximized split-half reliability estimates. Journal of Educational Measurement, 16, 89–99.
Cheung, O. S., Richler, J. J., Palmeri, T. J., & Gauthier, I. (2008). Revisiting the role of spatial frequencies in the holistic processing of faces. Journal of Experimental Psychology: Human Perception and Performance, 34, 1327–1336. doi:10.1037/a0011752
Chiou, J., & Spreng, R. A. (1996). The reliability of difference scores: A re-examination. Journal of Consumer Satisfaction, Dissatisfaction and Complaining Behavior, 9, 158–167.
Chua, K.-W., Richler, J. J., & Gauthier, I. (2014). Becoming a Lunari or Taiyo expert: Learner attention to parts drives holistic processing of faces. Journal of Experimental Psychology: Human Perception and Performance, 40, 1174–1182. doi:10.1037/a0035895
DeGutis, J., Wilmer, J., Mercado, R. J., & Cohan, S. (2013). Using regression to measure holistic face processing reveals a strong link with face recognition ability. Cognition, 126, 87–100. doi:10.1016/j.cognition.2012.09.004
Duchaine, B., & Nakayama, K. (2006). The Cambridge Face Memory Test: Results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic faces. Neuropsychologia, 44, 576–585.
Eide, P., Kemp, A., Silberstein, R. B., Nathan, P. J., & Stough, C. (2002). Test–retest reliability of the emotional Stroop task: Examining the paradox of measurement change. Journal of Psychology, 136, 514–520.
Gauthier, I., & Bukach, C. (2007). Should we reject the expertise hypothesis? Cognition, 103, 322–330. doi:10.1016/j.cognition.2006.05.003
Gauthier, I., Klaiman, C., & Schultz, R. T. (2009). Face composite effects reveal abnormal face processing in autism spectrum disorders. Vision Research, 49, 470–478.
Gauthier, I., Williams, P., Tarr, M. J., & Tanaka, J. (1998). Training “Greeble” experts: A framework for studying expert object recognition processes. Vision Research, 38, 2401–2428. doi:10.1016/S0042-6989(97)00442-2
Goffaux, V., & Rossion, B. (2006). Faces are “spatial”—Holistic face perception is supported by low spatial frequencies. Journal of Experimental Psychology: Human Perception and Performance, 32, 1023–1039. doi:10.1037/0096-1523.32.4.1023
Konar, Y., Bennett, P. J., & Sekuler, A. B. (2010). Holistic processing is not correlated with face-identification accuracy. Psychological Science, 21, 38–43.
Llabre, M. M., Spitzer, S. B., Saab, P. G., Ironson, G. H., & Schneiderman, N. (1991). The reliability and specificity of delta versus residulized change as measures of cardiovascular reactivity to behavioral challenges. Psychophysiology, 28, 701–711.
Malgady, R. G., & Colon-Malgady, G. (1991). Comparing the reliability of difference scores and residuals in analysis of covariance. Educational and Psychological Measurement, 51, 803–807.
McGugin, R. W., Richler, J. J., Herzmann, G., Speegle, M., & Gauthier, I. (2012). The Vanderbilt Expertise Test reveals domain-general and domain-specific sex effects in object recognition. Vision Research, 69, 10–22. doi:10.1016/j.visres.2012.07.014
McKone, E., Kanwisher, N., & Duchaine, B. C. (2007). Can generic expertise explain special processing for faces? Trends in Cognitive Sciences, 11, 8–15. doi:10.1016/j.tics.2006.11.002
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw–Hill.
Palermo, R., Willis, M. L., Rivolta, D., McKone, E., Wilson, E. C., & Calder, A. J. (2011). Impaired holistic coding of facial expression and facial identity in congenital prosopagnosia. Neuropsychologia, 49, 1226–1235.
Peter, J., Gilbert, A., Churchill, J., & Brown, T. J. (1993). Caution in the use of difference scores in consumer research. Journal of Consumer Research, 19, 662–665.
Richler, J. J., Bukach, C. M., & Gauthier, I. (2009). Context influences holistic processing of nonface objects in the composite task. Attention, Perception, & Psychophysics, 71, 530–540. doi:10.3758/APP.71.3.530
Richler, J. J., Cheung, O. S., & Gauthier, I. (2011a). Beliefs alter holistic face processing . . . if response bias is not taken into account. Journal of Vision, 11(13), 17:1–13. doi:10.1167/11.13.17
Richler, J. J., Cheung, O. S., & Gauthier, I. (2011b). Holistic processing predicts face recognition. Psychological Science, 22, 464–471. doi:10.1177/0956797611401753
Richler, J. J., & Gauthier, I. (in press). A meta-analysis and review of holistic processing. Psychological Bulletin.
Richler, J. J., Palmeri, T. J., & Gauthier, I. (2012). Meanings, mechanisms, and measures of holistic processing. Frontiers in Psychology, 3(553), 1–6. doi:10.3389/fpsyg.2012.00553
Richler, J. J., Tanaka, J. W., Brown, D. D., & Gauthier, I. (2008). Why does selective attention to parts fail in face processing? Journal of Experimental Psychology: Learning, Memory, and Cognition, 34, 1356–1368.
Richler, J. J., Wong, Y. K., & Gauthier, I. (2011d). Perceptual expertise as a shift from strategic interference to automatic holistic processing. Current Directions in Psychological Science, 20, 129–134. doi:10.1177/0963721411402472
Rogosa, D., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Quantitative Methods in Psychology, 92, 726–748.
Rossion, B. (2013). The composite face illusion: A whole window into our understanding of holistic face perception. Visual Cognition, 21, 139–253.
Schmidt, F. L., & Hunter, J. E. (1996). Measurement error in psychological research: Lessons from 26 research scenarios. Psychological Methods, 1, 199–223. doi:10.1037/1082-989X.1.2.199
Strauss, G. P., Allen, D. N., Jorgensen, M. L., & Cramer, S. L. (2005). Test–retest reliability of standard and emotional Stroop tasks an investigation of color–word and picture–word versions. Assessment, 12, 330–337.
Thompson, B. (1994). Guidelines for authors. Educational and Psychological Measurement, 54, 837–847.
Troje, N. F., & Bülthoff, H. H. (1996). Face recognition under varying poses: The role of texture and shape. Vision Research, 36, 1761–1771. doi:10.1016/0042-6989(95)00230-8
Wong, A. C.-N., Palmeri, T. J., & Gauthier, I. (2009). Conditions for facelike expertise with objects: Becoming a Ziggerin expert—but which type? Psychological Science, 20, 1108–1117. doi:10.1111/j.1467-9280.2009.02430.x
Young, A. W., Hellawell, D., & Hay, D. C. (1987). Configurational information in face perception. Perception, 16, 747–759.
Zimmerman, D. W., & Williams, R. H. (1998). Reliability scores under realistic assumptions about properties of pre-test and post-test scores. British Journal of Mathematical and Statistical Psychology, 51, 343–351.
Author note
This work was supported by the National Science Foundation (Grant No. SBE-0542013), Vanderbilt Vision Research Center (Grant No. P30-EY008126), and National Eye Institute (Grant No. R01 EY013441-06A2). We thank Riaun Floyd and Magen Speegle for their help with data collection and stimulus construction, and Jeremy Wilmer and Joseph DeGutis for helpful discussions.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ross, D.A., Richler, J.J. & Gauthier, I. Reliability of composite-task measurements of holistic face processing. Behav Res 47, 736–743 (2015). https://doi.org/10.3758/s13428-014-0497-4
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13428-014-0497-4