Abstract
Crossmodal spatial integration between auditory and visual stimuli is a common phenomenon in space perception. The principles underlying such integration have been outlined by neurophysiological and behavioral studies in animals; this study investigated whether the integrative effects observed in animals also apply to humans. In this experiment we systematically varied the spatial disparity (0°, 16°, and 32°) and the temporal interval (0, 100, 200, 300, 400, and 500 ms) between the visual and the auditory stimuli. Normal subjects were required to detect visual stimuli presented below threshold either in unimodal visual conditions or in crossmodal audiovisual conditions. Signal detection measures were used. An enhancement of the perceptual sensitivity (d′) for luminance detection was found when the audiovisual stimuli followed a simple spatial and temporal rule, governing multisensory integration at the neuronal level.
Similar content being viewed by others
References
Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of multisensory integration sites by application of electrophysiological criteria to the BOLD response. Neuroimage 14:427–438
Deubel H, Schneider WX (1996) Saccade target selection and object recognition: evidence from a common attentional mechanism. Vision Res 36:1827–1837
Driver J, Spence C (1998) Crossmodal attention. Curr Op Neurobiol 8:245–253
Driver J, Spence C (2000) Beyond modularity and convergence. Curr Biol 10: R731–R735
Duncan J (1980) The demonstration of capacity limitation. Cogn Psychol 12:75–96
Frassinetti F, Bolognini N, Làdavas E (2002a) Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147:332–343
Frassinetti F, Pavani F, Làdavas E (2002b) Acoustic vision of neglected stimuli: interaction among spatially converging audiovisual inputs in neglect patients. J Cogn Neurosci 14:62–64
Frens MA, Van Opstal AJ (1998) Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus. Brain Res Bull 46:211–224
Hoffman J, Subramaniam B (1995) The role of visual attention in saccadic eye movements. Percept Psychophys 57:787–795
Jay MF, Sparks DL (1987) Sensorimotor integration in the primate superior colliculus. II. Coordinates of auditory signals. J Neurophysiol 57:35–55
King AJ, Hutchings ME (1987) Spatial response property of acoustically responsive neurons in the superior colliculus of the ferret: a map of auditory space. J Neurophysiol 57:596–624
King AJ, Palmer AR (1985) Spatial response proprieties of visual and auditory information in bimodal neurons in the guinea-pig superior colliculus. Exp Brain Res 60:492–500
Knudsen EI (1982) Auditory and visual maps of space in the optic tectum of the owl. Neuroscience 2:1177–1194
Knudsen EI, Brainard MS (1995) Creating a unified representation of visual and auditory space in the brain. Annu Rev Neurosci 18:19–43
Kowler E, Anderson E, Dosher B, Blaser E (1995) The role of attention in programming saccades. Vision Res 35:1897–1916
Lovelace CT, Stein BE, Wallace MT (2003) An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Cognit Brain Res 17:447–453
Macaluso E, Frith C, Driver J (2001) Multisensory integration and crossmodal attention effects in the human brain. Science 292:1791a
McDonald JJ, Teder-Salerjarvi WA, Hillyard SA (2000) Involuntary orienting to a sound improves visual perception. Nature 407:906–908
Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221:389–391
Meredith MA, Stein BE (1986a) Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365:350–354
Meredith MA, Stein BE (1986b) Visual, auditory and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56:640–662
Meredith MA, Stein BE (1996) Spatial determinants of multisensory integration in cat superior colliculus neurons. J Neurophysiol 75:1843–1857
Meredith MA, Nemitz, JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 10:3215–3229
Middlebrooks JC, Knudsen EI (1984) A neural code for auditory space in the cat’s superior colliculus. J Neurosci 4:2621–2634
Newman EA, Hartline PH (1981) Integration of visual and infrared information in bimodal neurons in the rattlesnake optic tectum. Science 213:789–791
Odgaard EC, Arieh Y, Marks LE (2003) Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percept Psychophys 65:123–132
Populin LC, Yin TC (2002) Bimodal interactions in the superior colliculus of the behaving cat. J Neurosci 22:2826–2834
Robertson IH, Mattingley JB, Rorden C, Driver J (1998) Phasic alerting of neglect patients overcomes their spatial deficit in visual awareness. Nature 395:169–172
Rorden C, Driver J (1999) Does auditory attention shift in the direction of an upcoming saccade? Neuropsychologia 37:357–377
Shaw ML (1980) Identifying attentional and decision making components in information processing. In: Nickerson RS (eds) Attention and performance VIII. Erlbaum, Hillsdale, pp 277–296
Spence C, Driver J (1997) Audiovisual links in exogenous covert spatial orienting. Percept Psychophys 59:1–22
Stein BE, Meredith MA (1993) Merging of the senses. MIT Press, Cambridge
Stein BE, London N, Wilkinson LK, Price DD (1996) Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cogn Neurosci 8:497–506
Van de Par S, Kohlrausch A (2000) Sensitivity to auditory-visual asynchrony and the jitter in auditory-visual timing. In: Rogowitz BE, Pappas TN (eds) Human vision and electronic imaging V. Proceeding of the SPIE, vol 3959. SPIE Press, Bellingham, pp 234–242
Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76:1246–1266
Wallace MT, Meredith MA, Stein BE (1998) Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80:1006–1010
Bell AH, Corneil BD, Meredith MA, Munoz DP (2001) The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. Can J Exp Psychol 55:123–132
Peck CK (1987) Visual-auditory interactions in cat superior colliculus: their role in the control of gaze. Brain Res 420:162–166
Spence C (2001) Crossmodal attentional capture: a controversy resolved? In: Folks CL, Gibson BS (eds) Attraction, distraction, and action: multiple perspectives on attentional capture. Elsevier, Amsterdam
Acknowledgements
This work was supported by grants from MURST to E.L.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bolognini, N., Frassinetti, F., Serino, A. et al. “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs. Exp Brain Res 160, 273–282 (2005). https://doi.org/10.1007/s00221-004-2005-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-004-2005-z