Infants' intermodal perception of events

doi:10.1016/0010-0285(76)90018-9

Cognitive Psychology

Volume 8, Issue 4, October 1976, Pages 553-560

https://doi.org/10.1016/0010-0285(76)90018-9 Get rights and content

Abstract

Four-month-old infants viewed two sound motion picture films of simple, natural events. The films were projected side by side, as one of the two sound tracks was played through a centrally placed speaker. Infants' visual attention to the films was consistently influenced by what they heard: They looked primarily at the event specified by the sound track. The experiment demonstrates that infants are able to perceive relations between sights and sounds in the absence of spatial cues. They respond to a perceived intermodal invariance with increased attention to the event reaching them over two modalities.

References (7)

E. Aronson et al.
Space perception in early infancy: Perception within a common auditory—visual space
Science
(1971)
T.G.R. Bower
The object in the world of the infant
Scientific American
(1971)
T.G.R. Bower

There are more references available in the full text version of this article.

Cited by (183)

The practice of metaphor in conversation: an ecological integrational approach
2024, Language Sciences
In this study an integrational linguistic approach to metaphor is used in the context of an ecological psychology study of novel metaphor creation by adults in a structured conversation setting. This paper forms an example of the proposed complementarity of integrational linguistics (Harris, 1981) and ecological psychology (Gibson, 1979; Jones and Read, 2023) by providing a study of novel metaphor that eschews the traditional “coded carrier of message” assumptions about language, as well as the traditional “knowledge is mental representation” assumptions of representationalist cognitive psychology. Instead, novel metaphor is presented as the sine qua non of the creation of signs in the process of communication, and that creation is proposed to be founded on the perception of invariants across different naturally occurring kinds of objects and events (cf., Read & Szokolszky, 2016). The practice of metaphor is taken as a special case of perceptually guided kinematic semiology. This study describes the practice of metaphor by adults in a structured experimental situation designed to present metaphoric resemblance and to encourage the practice of metaphor with guiding verbal instructions. Such a study allows explication of the integrational method as applied to structured conversational settings. Although the conversation and context were designed to draw attention to metaphoric resemblance and to encourage verbal metaphor, not everyone practiced metaphor, showing that the practice is not determined by context. When metaphor was created, the form often mirrored the perceptual invariants available to the perceiver, i.e., motion or stationary resemblances. No one created exactly the same metaphor even in this consistent context, which emphasizes the creative aspect of metaphor as a prototype of sign creation, with its core properties of novelty, enhanced interest and noticeableness. I make the following central points: communicating by creating and integrating signs is the foundation of language; metaphor is the prototype of the creation of signs, the creation of novel metaphor in conversation is a practice that enhances communication, even in structured experimental settings; and conversation can be studied as the ongoing process of sign integration, that is, as perceptually guided kinematic semiology. The current study shows that metaphor as a practice in conversation is closely coordinated with the perception of metaphoric resemblance and the request to talk about objects and events that are alike metaphorically. Finally, it is argued that direct perception is the best approach on which to found an account of metaphor in communication.
Infant distractibility from social events mediates the relation between maternal responsiveness and infant language outcomes
2023, Infant Behavior and Development
Research demonstrates that contingent and appropriate maternal responsiveness to infant requests and bids for attention leads to better language outcomes. Research also indicates that infants who are less distracted by irrelevant competing stimulation and attend efficiently to audiovisual social events (e.g., faces and voices) show better language outcomes. However, few studies have assessed relations between maternal responsiveness, infant attention to faces and voices, and distractibility, and how together these factors lead to early language outcomes. A newly developed audiovisual protocol, the Multisensory Attention Assessment Protocol (MAAP; Bahrick et al., 2018), allows researchers to examine individual differences in attention to faces and voices and distractibility, and to assess relations with other variables. At 12 months, infants (n = 79) in an ongoing longitudinal study participated in the MAAP to assess intersensory matching of synchronous faces and voices and attention to an irrelevant competing visual distractor event. They also were observed in a brief play interaction to assess infant bids for attention and maternal responsiveness (accept, redirect, or ignore). At 18 months, receptive and expressive language were assessed using the Mullen Scales of Early Learning. Several noteworthy findings emerged: 1) mothers were generally responsive, accepting 74% and redirecting 14% of infant bids, 2) infants who had a greater number of their bids redirected by mothers, and who had better intersensory matching of synchronous faces and voices, showed less attention to the distractor, and 3) infants who showed less attention to the distractor had better receptive language. Findings demonstrate that maternal redirecting of infant attention by mothers who are generally responsive may promote better infant attentional control (lower distractibility) which in turn predicts better receptive language in toddlers.
Riding the elephant in the room: Towards a revival of the optimal level of stimulation model
2022, Developmental Review
Citation Excerpt :
These studies seem to indicate that, once infants identify an item as new, they may inhibit their exploration. Other evidence come from intermodal studies with infants looking longer at a visual stimulus that matches an auditory stimulus (more familiar) over one that mismatches it (Golinkoff et al., 1987; Spelke, 1976). In adults as well, congruent sounds increase the pleasantness of olfactory stimuli (Seo et al., 2014; Seo & Hummel, 2011).
Phenomena such as engagement, attention and curiosity rely heavily on the “optimal-level of stimulation (or arousal)” model, which suggests they are driven by stimuli being neither too simple nor too complex. Two points often overlooked in psychology are that each stimulus is simultaneously processed with its context, and that a stimulus complexity is relative to an individual’s cognitive resources to process it. According to the “optimal-level of stimulation” model, while familiar contexts may decrease the overall stimulation and favour exploration of novelty, a novel context may increase the overall stimulation and favour preference for familiarity. In order to stay closer to their optimum when stimulation is getting too high or too low, individuals can explore other stimuli, adopt a different processing style or be creative. The need and the ability to adopt such strategies will depend upon the cognitive resources available, which can be affected by contextual stimulation and by other factors such as age, mood or arousability. Drawing on empirical research in cognitive and developmental psychology, we provide here an updated “optimal-level of stimulation” model, which is holistic and coherent with previous literature. Once taken into account the role of contextual stimulation as well as the diverse factors influencing internal cognitive resources, such model fits with and enriches other existing theories related to exploratory behaviors. By doing so, it provides a useful framework to investigate proximate explanations underlying learning and cognitive development, and to develop future interventions related, for example, to eating, and learning disorders.
Motion or emotion: Infants discriminate emotional biological motion based on low-level visual information
2019, Infant Behavior and Development
Citation Excerpt :
In addition to perceptual discrimination, infants are capable of categorizing emotions in vocal tone by 5 months, and facial expression by 7 months (Flom & Bahrick, 2007). When the intermodal preference technique (Spelke, 1976) is used, 7-month-old infants can match emotions across face and voice (Walker, 1982), and infants as young as 3.5 months can do this when the face is that of their own mother, but not when the face is someone unfamiliar (Kahana-Kalman & Walker-Andrews, 2001; Walker-Andrews, Krogh-Jespersen, Mayhew, & Coffield, 2011). Seven-month-old infants are also capable of matching the emotions of a face and voice when the bottom third of the face is occluded (Walker-Andrews, 1986), and can categorize certain emotional expressions, such as surprise when tested against fear (Ludemann & Nelson, 1988).
Infants’ ability to discriminate emotional facial expressions and tones of voice is well-established, yet little is known about infant discrimination of emotional body movements. Here, we asked if 10–20-month-old infants rely on high-level emotional cues or low-level motion related cues when discriminating between emotional point-light displays (PLDs). In Study 1, infants viewed 18 pairs of angry, happy, sad, or neutral PLDs. Infants looked more at angry vs. neutral, happy vs. neutral, and neutral vs. sad. Motion analyses revealed that infants preferred the PLD with more total body movement in each pairing. Study 2, in which infants viewed inverted versions of the same pairings, yielded similar findings except for sad-neutral. Study 3 directly paired all three emotional stimuli in both orientations. The angry and happy stimuli did not significantly differ in terms of total motion, but both had more motion than the sad stimuli. Infants looked more at angry vs. sad, more at happy vs. sad, and about equally to angry vs. happy in both orientations. Again, therefore, infants preferred PLDs with more total body movement. Overall, the results indicate that a low-level motion preference may drive infants’ discrimination of emotional human walking motions.
M2CURL: Sample-Efficient Multimodal Reinforcement Learning via Self-Supervised Representation Learning for Robotic Manipulation
2024, arXiv
What, if anything, can be considered an amodal sensory dimension?
2024, Psychonomic Bulletin and Review

View all citing articles on Scopus

^☆: This research was supported by a Training Grant to Cornell University from the National Institute of Child Health and Human Development (TO1 HD00381-05).

View full text

Infants' intermodal perception of events☆

Abstract

Space perception in early infancy: Perception within a common auditory—visual space

Science

The object in the world of the infant

Scientific American