Abstract
We tested whether auditory sequences of beeps can modulate the tactile perception of sequences of taps (two to four taps per sequence) delivered to the index fingertip. In the first experiment, the auditory and tactile sequences were presented simultaneously. The number of beeps delivered in the auditory sequence were either the same as, less than, or more than the number of taps of the simultaneously presented tactile sequence. Though task-irrelevant (subjects were instructed to focus on the tactile stimuli), the auditory stimuli systematically modulated subjects’ tactile perception; in other words subjects’ responses depended significantly on the number of delivered beeps. Such modulation only occurred when the auditory and tactile stimuli were similar enough. In the second experiment, we tested whether the automatic auditory-tactile integration depends on simultaneity or whether a bias can be evoked when the auditory and tactile sequence are presented in temporal asynchrony. Audition significantly modulated tactile perception when the stimuli were presented simultaneously but this effect gradually disappeared when a temporal asynchrony was introduced between auditory and tactile stimuli. These results show that when provided with auditory and tactile sensory signals that are likely to be generated by the same stimulus, the central nervous system (CNS) tends to automatically integrate these signals.
Similar content being viewed by others
References
Alais D, Burr D (2004) The ventriloquist effect results from near-optimal bimodal integration. Curr Biol 14:257–262
Aschersleben G, Prinz W (1995) Synchronizing actions with events: the role of sensory information. Percept Psychophys 57:305–317
Aschersleben G, Drewing K, Stenneken P (2002) Temporal coordination of simple movements. Cognitive Processing 1:37–60
Bermant RI, Welch RB (1976) Effect of degree of separation of visual-auditory stimulus and eye position upon spatial interaction of vision and audition. Percept Motor Skill 42:487–493
Bernstein IH, Clark MH, Edelstein BA (1969) Effects of an auditory signal on visual reaction time. J Exp Psychol 80:567–569
Bertelson P, Radeau M (1981) Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept Psychophys 29:578–584
Desimone R, Gross CG (1979) Visual areas in the temporal cortex of the macaque. Brain Res 178:363–380
DiFranco DE, Beauregard GL, Srinivasan MA (1997) The effect of auditory cues on the haptic perception of stiffness in virtual environments. P ASME 61:17–22
Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429–433
Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169
Fendrich R, Corballis PM (2001) The temporal cross-capture of audition and vision. Percept Psychophys 63:719–725
Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE (2000) Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cognitive Brain Res 10:77–83
Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, Murray MM (2002) Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88:540–543
Fraisse P (1980) Les synchronizations sensori-motrices aux rythmes [The sensorimotor synchronization of rhythms]. In: Requin J (ed) Anticipation et comportement. Editions du CNRS, Paris, pp 233–257
Gepshtein S, Banks MS (2003) Viewing geometry determines how vision and haptics combine in size perception. Curr Biol 13:483–488
Gielen SCAM, Schmidt RA, van den Heuvel PJM (1983) On the nature of intersensory facilitation of reaction time. Percept Psychophys 34:161–168
Gobbelé R, Schürmann M, Forss N, Juottonen, K, Buchner H, Hari R (2003) Activation of the human posterior parietal and temporoparietal cortices during audiotactile interaction. NeuroImage 20:503–511
Guest S, Catmur C, Lloyd D, Spence C (2002) Audiotactile interactions in roughness perception. Exp Brain Res 146:161–171
Hershenson M (1962) Reaction time as a measure of intersensory facilitation. J Exp Psychol 63:289–293
Hikosaka K, Iwai E, Saito H, Tanaka K (1988) Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J Neurophysiol 60:1615–1637
Hötting K, Röder B (2004) Hearing cheats touch, but less in congenitally blind than in sighted individuals. Psychol Sci 15:60–64
Hyvärinen J, Poranen A (1974) Function of the parietal associative area 7 as revealed from cellular discharges in alert monkeys. Brain 97:673–692
Jousmäki V, Hari R (1998) Parchment-skin illusion: sound-biased touch. Curr Biol 8:190
Leinonen L, Hyvarinen J, Sovijarvi AR (1980) Functional properties of neurons in the temporo-parietal association cortex of awake monkey. Exp Brain Res 39:203–215
Lütkenhöner B, Lammertmann C, Simoes C, Hari R (2002) Magnetoencephalographic correlates of audiotactile interaction. NeuroImage 15:509–522
Morein-Zamir S, Soto-Faraco S, Kingstone A (2003) Auditory capture of vision: examining temporal ventriloquism. Cognitive Brain Res 17:154–163
Morell LK (1968) Temporal characteristics of sensory interaction in choice reaction times. J Exp Psychol 77:14–18
Nickerson RS (1973) Intersensory facilitation of reaction time: Energy summation or preparation enhancement? Psychol Rev 80:489–509
Shams L, Kamitani Y, Shimojo S (2000) What you see is what you hear. Nature 408:788
Shams L, Kamitani Y, Shimojo S (2002) Visual illusion induced by sound. Cognitive Brain Res 14:147–152
Slutsky D, Recanzone GH (2001) Temporal and spatial dependency of the ventriloquism effect. NeuroReport 12:7–10
Wu WC, Basdogan C, Srinivasan MA (1999) Visual, haptic, and bimodal perception of size and stiffness in virtual environments. P ASME 67:19–26
Acknowledgements
This work was supported by the Max-Planck Society and by the 5th Framework IST Program of the EU (IST-2001–38040, TOUCH-HapSys). We thank Kari Hoffman and Roland Fleming for helpful comments.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bresciani, JP., Ernst, M.O., Drewing, K. et al. Feeling what you hear: auditory signals can modulate tactile tap perception. Exp Brain Res 162, 172–180 (2005). https://doi.org/10.1007/s00221-004-2128-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-004-2128-2