Abstract
In the McGurk effect, perceptual identification of auditory speech syllables is influenced by simultaneous presentation of discrepant visible speech syllables. This effect has been found in subjects of different ages and with various native language backgrounds. But no McGurk tests have been conducted with prelinguistic infants. In the present series of experiments, 5-month-old English-exposed infants were tested for the McGurk effect. Infants were first gaze-habituated to an audiovisual /va/. Two different dishabituation stimuli were then presented: audio /ba/-visual /va/ (perceived by adults as /va/), and audio /da/-visual /va/ (perceived by adults as /da/). The infants showed generalization from the audiovisual /va/ to the audio /ba/-visual /va/ stimulus but not to the audio /da/-visual /va/ stimulus. Follow-up experiments revealed that these generalization differences were not due to a general preference for the audio /da/-visual /va/ stimulus or to the auditory similarity of /ba/ to /va/ relative to /da/. These results suggest that the infants were visually influenced in the same way as Englishspeaking adults are visually influenced.
Article PDF
Similar content being viewed by others
References
Aronson, E., &Rosenbloom, S. (1971). Space perception in early infancy: Perception within a common auditory-visual space.Science,172, 1161–1163.
Aslin, R. N. (1987). Perceptual development.Annual Review of Psychology,39, 435–473.
Best, C., McRoberts, G. W., &Sithole, N. M. (1988). Examination of perceptual organization for nonnative speech contrasts: Zulu click discrimination by English-speaking adults and infants.Journal of Experimental Psychology: Human Perception & Performance,14, 345–360.
Boliek, C., Green, K., Fohr, K., & Obrzut, J. (1996).Auditory-visual perception of speech in children with learning disabilities: The McGurk effect. Manuscript in preparation.
Demany, L., McKenzie, B., &Vurpillot, E. (1977). Rhythm perception in early infancy. In J. Oates & S. Sheldon (Eds.),Cognitive development in infancy (pp. 105–109). Hove, U.K.: Erlbaum.
Desjardins, R. N., & Werker, J. F. (1995, June–July).4-month-old infants notice both auditory and visual components of speech. Poster presented at the annual meeting of the American Psychological Society, New York.
Diehl, R. L., &Kluender, K. R. (1989). On the objects of speech perception.Ecological Psychology,1, 121–144.
Dodd, B. (1979). Lipreading in infants: Attention to speech presented in and out of synchrony.Cognitive Psychology,11, 478–484.
Eilers, R. E., &Minifie, F. D. (1975). Fricative discrimination in early infancy.Journal of Speech & Hearing Research,18, 158–167.
Eilers, R. E., Wilson, W. R., &Moore, J. M. (1977). Developmental changes in speech discrimination in three-, six-, and twelve-monthold infants.Journal of Speech & Hearing Research,20, 766–780.
Eimas, P. D., &Tartter, V. C. (1979). On the development of speech perception: Mechanisms and analogies. In H. W. Reese & L. P. Lipsitt (Eds.),Advances in child development and behavior (Vol. 13, pp. 50–76). New York: Academic Press.
Fodor, J. A. (1983).Modularity of mind. Cambridge, MA: MIT Press, Bradford Books.
Fowler, C. A., &Dekle, D. J. (1991). Listening with eye and hand: Cross-modal contributions to speech perception.Journal of Experimental Psychology: Human Perception & Performance,17, 816–828.
Fowler, C. A., &Rosenblum, L. D. (1991). Perception of the phonetic gesture. In I. G. Mattingly & M. Studdert-Kennedy (Eds.),Modularity and the motor theory (pp. 33–50). Hillsdale, NJ: Erlbaum.
Gibson, J. J. (1979).The ecological approach to visual perception. Boston: Houghton-Mifflin.
Green, K. P., Kuhl, P. K., Meltzoff, A. N., &Stevens, E. B. (1991). Integrating speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect.Perception & Psychophysics,50, 524–536.
Horowitz, F. D. (1975). Infant attention and discrimination: Methodological and substantive issues.Monographs of the Society for Research in Child Development,5, 1–15.
Horowitz, F. D., Paden, L., Bhana, K., &Self, P. (1972). An infantcontrol procedure for studying infant visual fixations.Developmental Psychology,7, 90.
Klatt, D. H. (1980). Software for a cascade/parallel formant synthesizer.Journal of the Acoustical Society of America,67, 971–994.
Kuhl, P. K. (1979). The perception of speech in early infancy. In N. J. Lass (Ed.),Speech and language: Advances in basic research and practice (pp. 1–47). New York: Academic Press.
Kuhl, P. K. (1987). Perception of speech and sound in early infancy. In P. Salapatek & L. Cohen (Eds.),Handbook of infant perception: Vol. 2. From perception to cognition (pp. 275–382). New York: Academic Press.
Kuhl, P. K., &Meltzoff, A. N. (1982). The bimodal development of speech in infancy.Science,218, 1138–1141.
Kuhl, P. K., &Meltzoff, A. N. (1984). The intermodal representation of speech in infants.Infant Behavior & Development,7, 361–381.
Kuhl, P. K., &Meltzoff, A. N. (1988). Speech as an intermodal object of perception. In A. Yonas (Ed.),Perceptual development in infancy: The Minnesota Symposia on Child Psychology (Vol. 20, pp. 235–266). Hillsdale, NJ: Erlbaum.
Kuhl, P. K., Williams, K. A., Lacerda, F., Stevens, K. N., &Lindblom, B. (1992). Linguistic experience alters phonetic perception in infants by 6 months of age.Science,255, 606–608.
Kuhl, P. K., Williams, K. A., &Meltzoff, A. N. (1991). Cross-modal speech perception in adults and infants using nonspeech auditory stimuli.Journal of Experimental Psychology: Human Perception & Performance,17, 829–840.
Legerstee, M. (1990). Infants use multimodal information to imitate speech sounds.Infant Behavior & Development,13, 343–354.
Levitt, A., Jusczyk, P. W., Murray, J., &Carden, G. (1988). Context-effects in two-month-old infants’ perception of labiodental/interdental fricative contrasts.Journal of Experimental Psychology: Human Perception & Performance,14, 361–368.
Liberman, A. M., &Mattingly, I. G. (1985). The motor theory of speech perception revised.Cognition,21, 1–36.
MacKain, K., Studdert-Kennedy, M., Spieker, S., &Stern, D. (1983). Infant intermodal speech perception is a left hemisphere function.Science,219, 1347–1349.
Massaro, D. W. (1984). Children’s perception of visual and auditory speech.Child Development,55, 1777–1788.
Massaro, D. W. (1987).Speech perception by ear and eye: A paradigm for psychological inquiry. Hillsdale, NJ: Erlbaum.
Massaro, D. W., &Cohen, M. M. (1990). Perception of synthesized audible and visible speech.Psychological Science,1, 55–63.
Massaro, D. W., Cohen, M. M., Gesi, A., Heredia, R., &Tsuzaki, M. (1993). Bimodal speech perception: An examination across languages.Journal of Phonetics,21, 445–478.
Massaro, D. W., Thompson, L. A., Barron, B., &Laren, E. (1986). Developmental changes in visual and auditory contributions to speech perception.Journal of Experimental Child Psychology,41, 93–113.
McGrath, M., &Summerfield, Q. (1985). Intermodal timing relations and audio-visual speech recognition by normal-hearing adults.Journal of the Acoustical Society of America,77, 678–685.
McGurk, H., &MacDonald, J. W. (1976). Hearing lips and seeing voices.Nature,264, 746–748.
Meltzoff, A. N., &Kuhl, P. K. (1994). Faces and speech: Intermodal processing of biologically relevant signals in infants and adults. In D. J. Lewkowitz & R. Lickliter (Eds.),The development of intersensory perception: Comparative perspectives (pp. 335–369). Hillsdale, NJ: Erlbaum.
Mills, A. E. (1987). The development of phonology in the blind child. In B. Dodd & R. Campbell (Eds.),Hearing by eye: The psychology of lip-reading (pp. 145–162). Hillsdale, NJ: Erlbaum.
Mills, A. E., &Theim, R. (1980). Auditory visual fusions and illusions in speech perception.Linguistische Berichte,6, 85–106.
Pickens, J., Field, T., Nawrocki, T., Martinez, A., Soutollo, D., &Gonzalez, J. (1994). Full-term and preterm infants’ perception of face-voice synchrony.Infant Behavior & Development,17, 447–455.
Reisberg, D., McLean, J., &Goldfield, A. (1987). Easy to hear but hard to understand: A lipreading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.),Hearing by eye: The psychology of lip-reading (pp. 97–114). Hillsdale, NJ: Erlbaum.
Repp, B. H., Manuel, S. Y., Liberman, A. M., & Studdert-Kennedy, M. (1983, November).Exploring the /ldMcGurk Effect.” Paper presented at the meeting of the Psychonomic Society, San Diego.
Rosenblum, L. D., Johnson, J. A.J. A., & Saldaña, H. M. (in press). Point-light facial displays enhance comprehension of speech in noise.Journal of Speech & Hearing Research.
Rosenblum, L. D., &Saldaña, H. M. (1992). Discrimination tests of visually influenced syllables.Perception & Psychophysics,52, 461–473.
Rosenblum, L. D., &Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception.Journal of Experimental Psychology: Human Perception & Performance,22, 318–331.
Saldaña, H. M., &Rosenblum, L. D. (1993). Visual influences on auditory pluck and bow judgments.Perception & Psychophysics,54, 406–416.
Saldaña, H. M., &Rosenblum, L. D. (1994). Selective adaptation in speech perception using a compelling audiovisual adaptor.Journal of the Acoustical Society of America,95, 3658–3661.
Sekiyama, K., &Tokhura, Y. (1991). McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility.Journal of the Acoustical Society of America,90, 1797–1805.
Sekiyama, K., &Tokhura, Y. (1993). Inter-language differences in the influence of visual cues in speech perception.Journal of Phonetics,21, 427–444.
Shaw, R., Turvey, M. T., &Mace, W. (1982). Ecological psychology: The consequence of a commitment to realism. In W. Weimer & D. Palmero (Eds.),Cognition and the symbolic processes (Vol. 2, pp. 3–51). Hove, U.K.: Erlbaum.
Snyder, R. T., &Pope, P. (1970). New norms for and an item analysis of the Wepman Test at the first grade, six-year-level.Perceptual & Motor Skills,31, 1007–1010.
Spelke, E. S., &Cortelyou, A. (1980). Perceptual aspects of social knowing: Looking and listening in infancy. In M. E. Lamb & L. R. Sherrod (Eds.),Infant social cognition (pp. 58–63). Hillsdale, NJ: Erlbaum.
Spelke, E. S., &Owsley, C. (1979). Intermodal exploration and knowledge in infancy.Infant Behavior & Development,2, 13–27.
Sumby, W. H., &Pollack, I. (1954). Visual contribution to speech intelligibility in noise.Journal of the Acoustical Society of America,26, 212–215.
Summerfield, Q., MacLeod, P., McGrath, M., &Brooke, N. M. (1989).Lips, teeth, and the benefits of lipreading. InA. W. Young &H. D. Ellis (Eds.),Handbook of research on face processing (pp. 223–233). New York: Elsevier.
Walker, A. S. (1982). Intermodal perception of expression behaviors by human infants.Journal of Experimental Child Psychology,33, 514–535.
Walton, G. E., &Bower, T. G. R. (1993). Amodal representation of speech in infants.Infant Behavior & Development,16, 233–243.
Werker, J. (1989). Becoming a native listener.American Scientist,77, 54–59.
Werker, J., &Tees, R. (1984). Cross-language speech perception: Evidence for perceptual reorganization during the first year of life.Infant Behavior & Development,7, 49–63.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by NSF Grant DBS-9212225 awarded to the first author and a grant from the Natural Sciences and Engineering Research Council of Canada to the second author.
Rights and permissions
About this article
Cite this article
Rosenblum, L.D., Schmuckler, M.A. & Johnson, J.A. The McGurk effect in infants. Perception & Psychophysics 59, 347–357 (1997). https://doi.org/10.3758/BF03211902
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03211902