ReviewMultisensory processing after a brain damage: Clues on post-injury crossmodal plasticity from neuropsychology
Highlights
► A brain lesion can disrupt the ability to integrate multisensory information. ► A brain lesion can induce abnormal multisensory experiences. ► Spared multisensory mechanisms can compensate for post-injury unisensory disorders. ► Post-injury crossmodal plasticity represents an attempt to reconnect the senses. ► Post-injury multisensory phenomena are mediated by large-scale brain networks.
Introduction
The ability to integrate information from different sensory systems is a fundamental property of the brain. The effective combination of sensory information is a perceptual strategy that can allow for enhanced identification of external stimuli and facilitate behavioural responses to sensory events. This is particularly true in instances where the input from one sensory modality is ambiguous. Furthermore, the presence of a complementary sensory component can even alter a percept entirely, as in the case of multisensory illusions (e.g., Alais et al., 2011, Calvert et al., 2004, Stein, 1998).
The human brain is endowed with different neural mechanisms that evaluate whether there is a concordance of information arriving through different sensory channels or whether these same signals give rise to conflict and thus should be processed separately. Much of our knowledge regarding multisensory processing in the human brain is derived from brain imaging studies (e.g., Calvert, 2001, Driver and Noesselt, 2008). More recently, non-invasive brain stimulation techniques have also informed us regarding the causal relations that exist between activity within a specific brain region and individual perceptual multisensory functions (Bolognini and Maravita, 2011).
The aim of this review is to provide an overview of the current status of the multisensory research in cognitive neuropsychology particularly within the domains of multisensory interactions in sensory, spatial and body perception. In neuropsychological research, there are typically two lines of investigation regarding multisensory processing. First, what can be considered the traditional lesion-behaviour approach, attempts to establish whether a lesion to a given brain area or network disrupts the ability to integrate multisensory cues, or if it induces joint deficits across different sensory modalities. This approach dates back to Luria's early proposals suggesting that cerebral sites adjacent to, and effectively connected with, modality-specific primary sensory areas are likely to subserve multisensory integration. So far, neuropsychological research has mainly explored the notion of a supramodal processor by investigating associations and dissociations across different sensory modalities.
However, as highlighted recently by Pourtois et al. (2005), multisensory integration can be viewed as a process that is altogether different from the presupposition of common processing resources or reliance on common supramodal representations. As such, multisensory integration cannot be addressed by simply juxtaposing results obtained from studies looking at unisensory processing separately. Rather, the question should be whether a brain lesion in a given area impairs the co-ordination of activity across different senses, or if it reduces the ability to combine multiple sources of information. In turn, these impairments can be assessed by insensitivities to well-known multisensory illusions or by the failure to enhance perception in the presence of redundant information, or even the emergence of abnormal multisensory experiences.
Finally, another line of the investigation focuses on the behavioural benefit induced by spared multisensory mechanisms. Viewed in this way, the question becomes whether multisensory mechanisms can compensate for a modality-specific perceptual or cognitive impairment as a result of a brain lesion and damage (Ladavas, 2008).
Previous reviews have focused on the traditional lesion-behaviour approach, looking for a neural substrate of supramodal processor (e.g., Luria, 1962, Pavani et al., 2003, Pavani et al., 2004). This line of investigation has provided support to theoretical models that consider some neuropsychological disorders as dependent on the disruption of high-level supramodal representations. For instance, lesions of the posterior parietal cortex (PPC) are crucially associated with the disruption of high-level supramodal spatial representations (e.g., Pavani et al., 2003, Brozzoli et al., 2006); some areas of the language network of the left hemisphere play a key role in the crossmodal binding of linguistic information, and it may also contain a supramodal phonological processor (e.g., Campbell et al., 1990, Schmid and Ziegler, 2006, Hamilton et al., 2006); damages to the anteromedial temporal lobe may lead to multisensory recognition disorders for both objects and faces (e.g., Taylor et al., 2006, Taylor et al., 2009, Hoover et al., 2010, Gentileschi et al., 2001, Gainotti et al., 2008). Overall, this neuropsychological evidence is in broad agreement with the neuroimaging literature (see below), which has demonstrated the role of: posterior parietal areas in multisensory space and body representation (e.g., Macaluso and Maravita, 2010), a left-lateralized network, comprising the superior temporal, inferior parietal and inferior frontal regions, in the processing of multisensory speech (e.g., Campbell, 2008, Calvert, 2001), a distributed temporal lobe network, including posterior superior temporal regions, the perirhinal cortex, the lateral occipital complex, the anterior intraparietal sulcus and the right insula, for crossmodal identification and object and face recognition (e.g., Amedi et al., 2005). So far, most of these neuropsychological studies lack of rigorous anatomo-clinical evidence, which therefore precludes the inference of strict associations between the extension/site of a lesion, the resultant multisensory disorder, and the neuroimaging results in healthy humans.
The present review will focus on evidence of disruption or abnormal integration of multisensory cues and on spared multisensory processing in patients with an acquired lesion to putative higher-order heteromodal areas or low-level modality-specific areas.
Section snippets
Multisensory perception
Sensory perception can be modified, both quantitatively and qualitatively, by multisensory integration. For instance, in healthy subjects visual sensitivity can be enhanced by the presence of an uninformative auditory or tactile stimulus (e.g., Bolognini et al., 2005a, Bolognini and Maravita, 2007). Even the presence of smells can facilitate the visual identification of an odour source (Seigneuric et al., 2010). These examples demonstrate that a reliable and robust interpretation of sensory
Impaired multisensory integration after brain damage
So far, multisensory processing in brain-damaged patients has focused mainly on joint impairments across different sensory modalities. That is to say, whether a lesion in a given area can induce the same impairment across different senses. For instance, patients with visual neglect often exhibit deficits in the detection of contralesional sounds, particularly in the presence of an ipsilesional competitor (Pavani et al., 2003, Brozzoli et al., 2006). Unilateral spatial neglect is characterized
Conclusions and future directions
So far, neuropsychological studies of multisensory perception have characterized the disruptive effect of lesions within low-level visual or heteromodal association areas on the ability to combine spatial audio–visual and visuo-tactile information. More current evidence suggests that integration of spatially conflicting audio–visual stimuli invokes a mechanism that mainly implements the geniculo-striate circuit which is abolished by lesions localized to low-level (i.e., early) visual processing
Acknowledgments
This work was supported by grants from the University of Milano-Bicocca (F.A.R. 2011) to N.B. and an NIH/NEI RO1 GRANT EY019924 to L.B.M.
References (116)
- et al.
The posterior parietal cortex remaps touch into external space
Current Biology
(2010) The quest for the ‘critical lesion site’ in cognitive deficits: problems and perspectives
Cortex
(2011)- et al.
Ventriloquism in patients with unilateral visual neglect
Neuropsychologia
(2000) - et al.
Out-of-body experience, heautoscopy, and autoscopic hallucination of neurological origin implications for neurocognitive mechanisms of corporeal awareness and self-consciousness
Brain Research. Brain Research Reviews
(2005) - et al.
Visual localization of sounds
Neuropsychologia
(2005) - et al.
Proprioceptive alignment of visual and somatosensory maps in the posterior parietal cortex
Current Biology
(2007) - et al.
Auditory enhancement of visual phosphene perception: the effect of temporal and spatial factors and of stimulus intensity
Neuroscience Letters
(2010) - et al.
Neural basis of the ventriloquist illusion
Current Biology
(2007) - et al.
A hand in blindsight: hand placement near target improves size perception in the blind visual field
Neuropsychologia
(2008) - et al.
Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex
Current Biology
(2000)
Experimental remission of unilateral spatial neglect
Neuropsychologia
Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments
Neuron
Merging the senses into a robust percept
Trends in Cognitive Sciences
Cross-modal recognition disorders for persons and other unique entities in a patient with right fronto-temporal degeneration
Cortex
Is neocortex essentially multisensory?
Trends in Cognitive Sciences
A system of multimodal areas in the primate brain
Neuron
Spatial cognition: evidence from visual neglect
Trends in Cognitive Sciences
An acquired deficit of audiovisual speech processing
Brain and Language
Three hands: fragmentation of human bodily awareness
Neuroscience Letters
Multisensory integration: space, time and superadditivity
Current Biology
Superior voice recognition in a patient with acquired prosopagnosia and object agnosia
Neuropsychologia
Neurocognitive mechanisms of synaesthesia
Neuron
Multisensory mechanisms in temporo-parietal cortex support self-location and first-person perspective
Neuron
Neglect: a multisensory deficit?
Neuropsychologia
Modulation and rehabilitation of spatial neglect by sensory stimulation
Progress in Brain Research
Rehabilitation of neglect: an update
Neuropsychologia
Visuo-tactile representation of near-the-body space
Journal of Physiology, Paris
The representation of space near the body through touch and vision
Neuropsychologia
The claustrum/insula region integrates conceptually related sounds and pictures
Neuroscience Letters
Plasticity
NeuroRx
Auditory and multisensory aspects of visuospatial neglect
Trends in Cognitive Sciences
Auditory deficits in visuospatial neglect patients
Cortex
Perception of facial expressions and voices and of their combination in the human brain
Cortex
Multisensory convergence in calcarine visual areas in macaque monkey
International Journal of Psychophysiology
Parietal cortex and spatial cognition
Behavioural Brain Research
Crossmodal interactions: lessons from synesthesia
Progress in Brain Research
Audio–visual matching of speech and non-speech oral gestures in patients with aphasia and apraxia of speech
Neuropsychologia
Multisensory contributions to low-level, ‘unisensory’ processing
Current Opinion in Neurobiology
Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey
Brain Research
Can vision of the body ameliorate impaired somatosensory function?
Neuropsychologia
Multisensory processing in review: from physiology to behaviour
Seeing Perceiving
Functional imaging of human crossmodal identification and object recognition
Experimental Brain Research
Seeing the body modulates audiotactile integration
European Journal of Neuroscience
Neural substrates of sound-touch synesthesia after a thalamic lesion
Journal of Neuroscience
The influence of stimulus properties on multisensory processing in the awake primate superior colliculus
Canadian Journal of Experimental Psychology
Autoscopy in hemianopic field
Journal of Neurology, Neurosurgery and Psychiatry
Out-of-body experience and autoscopy of neurological origin
Brain
The out-of-body experience: disturbed self-processing at the temporo-parietal junction
Neuroscientist
Acoustical vision of below threshold stimuli: interaction among spatially converging audiovisual inputs
Experimental Brain Research
Visual search improvement in hemianopic patients after audio–visual stimulation
Brain
Cited by (41)
Effects of age and left hemisphere lesions on audiovisual integration of speech
2020, Brain and LanguageCitation Excerpt :Understanding how brain lesions impact multisensory functions provides critical causal evidence that can help elucidate typical brain processes and potentially inform treatment approaches for people with speech processing deficits (Bolognini, Convento, Rossetti, & Merabet, 2013; Fridriksson et al., 2009).
Hemianopia, spatial neglect, and their multisensory rehabilitation
2019, Multisensory Perception: From Laboratory to Clinic