Skip to main content
Log in

“Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

Crossmodal spatial integration between auditory and visual stimuli is a common phenomenon in space perception. The principles underlying such integration have been outlined by neurophysiological and behavioral studies in animals; this study investigated whether the integrative effects observed in animals also apply to humans. In this experiment we systematically varied the spatial disparity (0°, 16°, and 32°) and the temporal interval (0, 100, 200, 300, 400, and 500 ms) between the visual and the auditory stimuli. Normal subjects were required to detect visual stimuli presented below threshold either in unimodal visual conditions or in crossmodal audiovisual conditions. Signal detection measures were used. An enhancement of the perceptual sensitivity (d′) for luminance detection was found when the audiovisual stimuli followed a simple spatial and temporal rule, governing multisensory integration at the neuronal level.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of multisensory integration sites by application of electrophysiological criteria to the BOLD response. Neuroimage 14:427–438

    Article  CAS  PubMed  Google Scholar 

  • Deubel H, Schneider WX (1996) Saccade target selection and object recognition: evidence from a common attentional mechanism. Vision Res 36:1827–1837

    Article  CAS  PubMed  Google Scholar 

  • Driver J, Spence C (1998) Crossmodal attention. Curr Op Neurobiol 8:245–253

    Google Scholar 

  • Driver J, Spence C (2000) Beyond modularity and convergence. Curr Biol 10: R731–R735

    Article  CAS  PubMed  Google Scholar 

  • Duncan J (1980) The demonstration of capacity limitation. Cogn Psychol 12:75–96

    Article  Google Scholar 

  • Frassinetti F, Bolognini N, Làdavas E (2002a) Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147:332–343

    Article  PubMed  Google Scholar 

  • Frassinetti F, Pavani F, Làdavas E (2002b) Acoustic vision of neglected stimuli: interaction among spatially converging audiovisual inputs in neglect patients. J Cogn Neurosci 14:62–64

    Article  PubMed  Google Scholar 

  • Frens MA, Van Opstal AJ (1998) Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus. Brain Res Bull 46:211–224

    Article  CAS  PubMed  Google Scholar 

  • Hoffman J, Subramaniam B (1995) The role of visual attention in saccadic eye movements. Percept Psychophys 57:787–795

    CAS  PubMed  Google Scholar 

  • Jay MF, Sparks DL (1987) Sensorimotor integration in the primate superior colliculus. II. Coordinates of auditory signals. J Neurophysiol 57:35–55

    CAS  PubMed  Google Scholar 

  • King AJ, Hutchings ME (1987) Spatial response property of acoustically responsive neurons in the superior colliculus of the ferret: a map of auditory space. J Neurophysiol 57:596–624

    CAS  PubMed  Google Scholar 

  • King AJ, Palmer AR (1985) Spatial response proprieties of visual and auditory information in bimodal neurons in the guinea-pig superior colliculus. Exp Brain Res 60:492–500

    CAS  PubMed  Google Scholar 

  • Knudsen EI (1982) Auditory and visual maps of space in the optic tectum of the owl. Neuroscience 2:1177–1194

    CAS  PubMed  Google Scholar 

  • Knudsen EI, Brainard MS (1995) Creating a unified representation of visual and auditory space in the brain. Annu Rev Neurosci 18:19–43

    Article  CAS  PubMed  Google Scholar 

  • Kowler E, Anderson E, Dosher B, Blaser E (1995) The role of attention in programming saccades. Vision Res 35:1897–1916

    Article  CAS  PubMed  Google Scholar 

  • Lovelace CT, Stein BE, Wallace MT (2003) An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Cognit Brain Res 17:447–453

    Article  Google Scholar 

  • Macaluso E, Frith C, Driver J (2001) Multisensory integration and crossmodal attention effects in the human brain. Science 292:1791a

    Article  Google Scholar 

  • McDonald JJ, Teder-Salerjarvi WA, Hillyard SA (2000) Involuntary orienting to a sound improves visual perception. Nature 407:906–908

    Article  CAS  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221:389–391

    CAS  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1986a) Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365:350–354

    Article  CAS  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1986b) Visual, auditory and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56:640–662

    CAS  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1996) Spatial determinants of multisensory integration in cat superior colliculus neurons. J Neurophysiol 75:1843–1857

    CAS  PubMed  Google Scholar 

  • Meredith MA, Nemitz, JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 10:3215–3229

    Google Scholar 

  • Middlebrooks JC, Knudsen EI (1984) A neural code for auditory space in the cat’s superior colliculus. J Neurosci 4:2621–2634

    CAS  PubMed  Google Scholar 

  • Newman EA, Hartline PH (1981) Integration of visual and infrared information in bimodal neurons in the rattlesnake optic tectum. Science 213:789–791

    CAS  PubMed  Google Scholar 

  • Odgaard EC, Arieh Y, Marks LE (2003) Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percept Psychophys 65:123–132

    PubMed  Google Scholar 

  • Populin LC, Yin TC (2002) Bimodal interactions in the superior colliculus of the behaving cat. J Neurosci 22:2826–2834

    CAS  PubMed  Google Scholar 

  • Robertson IH, Mattingley JB, Rorden C, Driver J (1998) Phasic alerting of neglect patients overcomes their spatial deficit in visual awareness. Nature 395:169–172

    Article  CAS  PubMed  Google Scholar 

  • Rorden C, Driver J (1999) Does auditory attention shift in the direction of an upcoming saccade? Neuropsychologia 37:357–377

    Article  CAS  PubMed  Google Scholar 

  • Shaw ML (1980) Identifying attentional and decision making components in information processing. In: Nickerson RS (eds) Attention and performance VIII. Erlbaum, Hillsdale, pp 277–296

  • Spence C, Driver J (1997) Audiovisual links in exogenous covert spatial orienting. Percept Psychophys 59:1–22

    CAS  PubMed  Google Scholar 

  • Stein BE, Meredith MA (1993) Merging of the senses. MIT Press, Cambridge

  • Stein BE, London N, Wilkinson LK, Price DD (1996) Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. J Cogn Neurosci 8:497–506

    Google Scholar 

  • Van de Par S, Kohlrausch A (2000) Sensitivity to auditory-visual asynchrony and the jitter in auditory-visual timing. In: Rogowitz BE, Pappas TN (eds) Human vision and electronic imaging V. Proceeding of the SPIE, vol 3959. SPIE Press, Bellingham, pp 234–242

  • Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76:1246–1266

    CAS  PubMed  Google Scholar 

  • Wallace MT, Meredith MA, Stein BE (1998) Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80:1006–1010

    CAS  PubMed  Google Scholar 

  • Bell AH, Corneil BD, Meredith MA, Munoz DP (2001) The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. Can J Exp Psychol 55:123–132

    CAS  PubMed  Google Scholar 

  • Peck CK (1987) Visual-auditory interactions in cat superior colliculus: their role in the control of gaze. Brain Res 420:162–166

    Article  CAS  PubMed  Google Scholar 

  • Spence C (2001) Crossmodal attentional capture: a controversy resolved? In: Folks CL, Gibson BS (eds) Attraction, distraction, and action: multiple perspectives on attentional capture. Elsevier, Amsterdam

Download references

Acknowledgements

This work was supported by grants from MURST to E.L.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Elisabetta Làdavas.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bolognini, N., Frassinetti, F., Serino, A. et al. “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs. Exp Brain Res 160, 273–282 (2005). https://doi.org/10.1007/s00221-004-2005-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-004-2005-z

Keywords

Navigation