Skip to main content
Log in

Feeling what you hear: auditory signals can modulate tactile tap perception

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

We tested whether auditory sequences of beeps can modulate the tactile perception of sequences of taps (two to four taps per sequence) delivered to the index fingertip. In the first experiment, the auditory and tactile sequences were presented simultaneously. The number of beeps delivered in the auditory sequence were either the same as, less than, or more than the number of taps of the simultaneously presented tactile sequence. Though task-irrelevant (subjects were instructed to focus on the tactile stimuli), the auditory stimuli systematically modulated subjects’ tactile perception; in other words subjects’ responses depended significantly on the number of delivered beeps. Such modulation only occurred when the auditory and tactile stimuli were similar enough. In the second experiment, we tested whether the automatic auditory-tactile integration depends on simultaneity or whether a bias can be evoked when the auditory and tactile sequence are presented in temporal asynchrony. Audition significantly modulated tactile perception when the stimuli were presented simultaneously but this effect gradually disappeared when a temporal asynchrony was introduced between auditory and tactile stimuli. These results show that when provided with auditory and tactile sensory signals that are likely to be generated by the same stimulus, the central nervous system (CNS) tends to automatically integrate these signals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2A, B
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Alais D, Burr D (2004) The ventriloquist effect results from near-optimal bimodal integration. Curr Biol 14:257–262

    Article  CAS  PubMed  Google Scholar 

  • Aschersleben G, Prinz W (1995) Synchronizing actions with events: the role of sensory information. Percept Psychophys 57:305–317

    CAS  PubMed  Google Scholar 

  • Aschersleben G, Drewing K, Stenneken P (2002) Temporal coordination of simple movements. Cognitive Processing 1:37–60

    Google Scholar 

  • Bermant RI, Welch RB (1976) Effect of degree of separation of visual-auditory stimulus and eye position upon spatial interaction of vision and audition. Percept Motor Skill 42:487–493

    CAS  Google Scholar 

  • Bernstein IH, Clark MH, Edelstein BA (1969) Effects of an auditory signal on visual reaction time. J Exp Psychol 80:567–569

    CAS  PubMed  Google Scholar 

  • Bertelson P, Radeau M (1981) Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept Psychophys 29:578–584

    CAS  PubMed  Google Scholar 

  • Desimone R, Gross CG (1979) Visual areas in the temporal cortex of the macaque. Brain Res 178:363–380

    Article  CAS  PubMed  Google Scholar 

  • DiFranco DE, Beauregard GL, Srinivasan MA (1997) The effect of auditory cues on the haptic perception of stiffness in virtual environments. P ASME 61:17–22

    Google Scholar 

  • Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429–433

    Article  CAS  PubMed  Google Scholar 

  • Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169

    Article  PubMed  Google Scholar 

  • Fendrich R, Corballis PM (2001) The temporal cross-capture of audition and vision. Percept Psychophys 63:719–725

    CAS  PubMed  Google Scholar 

  • Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE (2000) Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cognitive Brain Res 10:77–83

    Article  CAS  Google Scholar 

  • Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, Murray MM (2002) Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88:540–543

    PubMed  Google Scholar 

  • Fraisse P (1980) Les synchronizations sensori-motrices aux rythmes [The sensorimotor synchronization of rhythms]. In: Requin J (ed) Anticipation et comportement. Editions du CNRS, Paris, pp 233–257

  • Gepshtein S, Banks MS (2003) Viewing geometry determines how vision and haptics combine in size perception. Curr Biol 13:483–488

    Article  CAS  PubMed  Google Scholar 

  • Gielen SCAM, Schmidt RA, van den Heuvel PJM (1983) On the nature of intersensory facilitation of reaction time. Percept Psychophys 34:161–168

    CAS  PubMed  Google Scholar 

  • Gobbelé R, Schürmann M, Forss N, Juottonen, K, Buchner H, Hari R (2003) Activation of the human posterior parietal and temporoparietal cortices during audiotactile interaction. NeuroImage 20:503–511

    Article  PubMed  Google Scholar 

  • Guest S, Catmur C, Lloyd D, Spence C (2002) Audiotactile interactions in roughness perception. Exp Brain Res 146:161–171

    Article  PubMed  Google Scholar 

  • Hershenson M (1962) Reaction time as a measure of intersensory facilitation. J Exp Psychol 63:289–293

    CAS  PubMed  Google Scholar 

  • Hikosaka K, Iwai E, Saito H, Tanaka K (1988) Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60:1615–1637

    CAS  PubMed  Google Scholar 

  • Hötting K, Röder B (2004) Hearing cheats touch, but less in congenitally blind than in sighted individuals. Psychol Sci 15:60–64

    Article  PubMed  Google Scholar 

  • Hyvärinen J, Poranen A (1974) Function of the parietal associative area 7 as revealed from cellular discharges in alert monkeys. Brain 97:673–692

    PubMed  Google Scholar 

  • Jousmäki V, Hari R (1998) Parchment-skin illusion: sound-biased touch. Curr Biol 8:190

    Article  Google Scholar 

  • Leinonen L, Hyvarinen J, Sovijarvi AR (1980) Functional properties of neurons in the temporo-parietal association cortex of awake monkey. Exp Brain Res 39:203–215

    CAS  PubMed  Google Scholar 

  • Lütkenhöner B, Lammertmann C, Simoes C, Hari R (2002) Magnetoencephalographic correlates of audiotactile interaction. NeuroImage 15:509–522

    Article  PubMed  Google Scholar 

  • Morein-Zamir S, Soto-Faraco S, Kingstone A (2003) Auditory capture of vision: examining temporal ventriloquism. Cognitive Brain Res 17:154–163

    Article  Google Scholar 

  • Morell LK (1968) Temporal characteristics of sensory interaction in choice reaction times. J Exp Psychol 77:14–18

    PubMed  Google Scholar 

  • Nickerson RS (1973) Intersensory facilitation of reaction time: Energy summation or preparation enhancement? Psychol Rev 80:489–509

    CAS  PubMed  Google Scholar 

  • Shams L, Kamitani Y, Shimojo S (2000) What you see is what you hear. Nature 408:788

    Article  CAS  PubMed  Google Scholar 

  • Shams L, Kamitani Y, Shimojo S (2002) Visual illusion induced by sound. Cognitive Brain Res 14:147–152

    Article  Google Scholar 

  • Slutsky D, Recanzone GH (2001) Temporal and spatial dependency of the ventriloquism effect. NeuroReport 12:7–10

    Article  CAS  PubMed  Google Scholar 

  • Wu WC, Basdogan C, Srinivasan MA (1999) Visual, haptic, and bimodal perception of size and stiffness in virtual environments. P ASME 67:19–26

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Max-Planck Society and by the 5th Framework IST Program of the EU (IST-2001–38040, TOUCH-HapSys). We thank Kari Hoffman and Roland Fleming for helpful comments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jean-Pierre Bresciani.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bresciani, JP., Ernst, M.O., Drewing, K. et al. Feeling what you hear: auditory signals can modulate tactile tap perception. Exp Brain Res 162, 172–180 (2005). https://doi.org/10.1007/s00221-004-2128-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-004-2128-2

Keywords

Navigation