Trends in Cognitive Sciences
ReviewEmotion Perception from Face, Voice, and Touch: Comparisons and Convergence
Section snippets
Nonverbal Emotions: Moving from a Unimodal to a Multimodal Perspective
Emotion (see Glossary) perception plays a ubiquitous role in human interactions and is hence of interest to a range of disciplines including psychology, psychiatry, and social neuroscience. However, its study has been dominated by facial emotions, with other modalities explored less frequently and often anchored within a framework derived from what we know about vision. Here we take a step back and put facial emotions on a par with vocal and tactile emotions. Like a frightened face, a shaking
Sensory Modalities for Emotion Expression
There is vigorous debate about what exactly individuals can express nonverbally. There are open questions about whether people are in an objective sense ‘expressing emotions’ as opposed to engaging in more strategic social communication, whether they express discrete emotions and which ones, and whether their expressions are culturally universal. For practical reasons we ignore these debates here and simply assume that people do express and perceive what we usually call emotions (Box 1).
Facial
Neural Systems for Perceiving Emotions
The neural systems underpinning the perception of emotions have been studied with a range of tasks and techniques. The following review emphasizes approaches that were used frequently and consistently across modalities and hence support a systematic comparison between face, voice, and touch. With respect to tasks, those contrasting nonverbal expressions with control stimuli (e.g., face–house) as well emotional with neutral expressions were selected as most relevant in our review. With respect
Modality Similarities and Differences
To date, research on emotion perception has emphasized the face. Moreover, insights from the face have served as a general guide in the search for and establishment of analogies with other modalities 72, 73 – and indeed, such analogies exist. Vision, audition, and touch each have slow and fast processing pathways that project to specialized regions of the sensory cortex, with upstream regions generally responding more robustly to social over nonsocial signals, especially if they are emotional (
Concluding Remarks
Social interactions elicit spontaneous and strategic expressions that we commonly classify as emotions and that profoundly influence a perceiver’s mental state and ongoing behavior 13, 57, 71. Neuroscience investigations have been uneven, with a historical overemphasis on facial signals. Although many questions remain (see Outstanding Questions), enough evidence has been accumulated to characterize mechanisms that are both unique and shared across modalities. Furthermore, although each channel
Glossary
- Affect
- often assumed to be a precursor to or prerequisite for a more differentiated emotion, affect is typically described as two dimensional, varying in valence (pleasant to unpleasant) and arousal (relaxed to excited). While applied primarily to the structure of feelings, it can also be applied to behavioral consequences of emotion states (e.g., approach–withdrawal).
- Amygdala
- an almond-shaped nucleus situated in the anterior aspect of the medial temporal lobe. It comprises several subnuclei and
References (111)
Evolution of the brainstem orofacial motor system in primates: a comparative study of trigeminal, facial, and hypoglossal nuclei
J. Hum. Evol.
(2005)The social role of touch in humans and primates: behavioural function and neurobiological mechanisms
Neurosci. Biobehav. Rev.
(2010)- et al.
Integrating simulation and theory of mind: from self to social cognition
Trends Cogn. Sci.
(2007) - et al.
Distributed and interactive brain mechanisms during emotion face perception: evidence from functional neuroimaging
Neuropsychologia
(2007) The primate amygdala in social perception – insights from electrophysiological recordings and stimulation
Trends Neurosci.
(2015)Lateralization for dynamic facial expressions in human superior temporal sulcus
Neuroimage
(2015)- et al.
Neural representations of emotion are organized around abstract event features
Curr. Biol.
(2015) Investigating the brain basis of facial expression perception using multi-voxel pattern analysis
Cortex
(2015)- et al.
Intact recognition of emotional prosody following amygdala damage
Neuropsychologia
(1999) N170 sensitivity to facial expression: a meta-analysis
Neurosci. Biobehav. Rev.
(2015)
Unraveling the biology of auditory learning: a cognitive–sensorimotor–reward framework
Trends Cogn. Sci.
The human voice areas: spatial organization and inter-individual variability in temporal and extra-temporal cortices
Neuroimage
On the spatial organization of sound processing in the human temporal lobe: a meta-analysis
Neuroimage
Impact of personality on the cerebral processing of emotional prosody
Neuroimage
Decoding of emotional information in voice-sensitive cortices
Curr. Biol.
On how the brain decodes vocal cues about speaker confidence
Cortex
Interpersonal stroking touch is targeted to C tactile afferent activation
Behav. Brain Res.
Thinking the voice: neural correlates of voice perception
Trends Cogn. Sci.
The socio-temporal brain: connecting people in time
Trends Cogn. Sci.
Audiovisual integration of emotional signals in voice and face: an event-related fMRI study
Neuroimage
Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing
Trends Cogn. Sci.
The overlapping relationship between emotion perception and theory of mind
Neuropsychologia
An integrative neural model of social perception, action observation, and theory of mind
Neurosci. Biobehav. Rev.
The functional role of the frontal cortex in pre-attentive auditory change detection
Neuroimage
Cortical control of facial expression
J. Comp. Neurol.
Measuring facial movement
Environ. Psychol. Nonverbal Behav.
Facial expressions of emotion are not culturally universal
Proc. Natl. Acad. Sci. U. S. A.
Perceptions of emotion from facial expressions are not culturally universal: evidence from a remote culture
Emotion
Cultural relativity in perceiving emotion from vocalizations
Psychol. Sci.
Processing emotional body expressions: state-of-the-art
Soc. Neurosci.
Acoustic profiles in vocal emotion expression
J. Pers. Soc. Psychol.
Path models of vocal emotion communication
PLoS One
A dimensional approach to vocal expression of emotion
Cogn. Emot.
The contribution of phonation type to the perception of vocal emotions in German: an articulatory synthesis study
J. Acoust. Soc. Am.
The voice conveys emotion in ten globalized cultures and one remote village in Bhutan
Emotion
Frequency of maternal touch predicts resting activity and connectivity of the developing social brain
Cereb. Cortex
Human C-tactile afferents are tuned to the temperature of a skin-stroking caress
J. Neurosci.
Coding of pleasant touch by unmyelinated afferents in humans
Nat. Neurosci.
The communication of emotion via touch
Emotion
Mechanisms of face perception
Annu. Rev. Neurosci.
The fusiform face area: a module in human extrastriate cortex specialized for face perception
J. Neurosci.
A role for somatosensory cortices in the visual recognition of emotion as revealed by three-dimensional lesion mapping
J. Neurosci.
Neurons in the human amygdala selective for perceived emotion
Proc. Natl Acad. Sci. U. S. A.
A leftward bias however you look at it: revisiting the emotional chimeric face task as a tool for measuring emotion lateralization
Laterality
Supramodal representations of perceived emotions in the human brain
J. Neurosci.
Multivariate neural biomarkers of emotional states are categorically distinct
Soc. Cogn. Affect. Neurosci.
What does the amygdala contribute to social cognition?
Ann. N. Y. Acad. Sci.
Neuropsychology of fear and loathing
Nat. Rev. Neurosci.
Electrophysiological studies of face perception in humans
J. Cogn. Neurosci.
Perceiving emotions in neutral faces: expression processing is biased by affective person knowledge
Soc. Cogn. Affect. Neurosci.
Cited by (227)
The role of affective touch in modulating emotion processing among preschool children
2023, Journal of Experimental Child PsychologyEncoding of multi-modal emotional information via personalized skin-integrated wireless facial interface
2024, Nature CommunicationsFacial and Body Posture Emotion Identification in Deaf and Hard-of-Hearing Young Adults
2024, Journal of Nonverbal Behavior