Skip to main content
SpringerLink
Log in

Hearing the speed: visual motion biases the perception of auditory tempo

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

The coupling between sensory and motor processes has been established in various scenarios: for example, the perception of auditory rhythm entails an audiomotor representation of the sounds. Similarly, visual action patterns can also be represented via a visuomotor transformation. In this study, we tested the hypothesis that the visual motor information, such as embedded in a coherent motion flow, can interact with the perception of a motor-related aspect in auditory rhythm: the tempo. In the first two experiments, we employed an auditory tempo judgment task where participants listened to a standard auditory sequence while concurrently watching visual stimuli of different motion information, after which they judged the tempo of a comparison sequence related to the standard. In Experiment 1, we found that the same auditory tempo was perceived as faster when it was accompanied by accelerating visual motion than by non-motion luminance change. In Experiment 2, we compared the perceived auditory tempo among three visual motion conditions, increase in speed, decrease in speed, and no speed change, and found the corresponding bias in judgment of auditory tempo: faster than it was, slower than it was, and no bias. In Experiment 3, the perceptual bias induced by the change in motion speed was consistently reflected in the tempo reproduction task. Taken together, these results indicate that between a visual spatiotemporal and an auditory temporal stimulation, the embedded motor representations from each can interact across modalities, leading to a spatial-to-temporal bias. This suggests that the perceptual process in one modality can incorporate concurrent motor information from cross-modal sensory inputs to form a coherent experience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. In order to facilitate the percept of forward motion, during instruction the experimenter mentioned that the appearance of motion condition might resemble that of the situation while they were sitting in a forward-moving car, with the scenes outside moving towards them. All participants were able to relate to this sensation while watching visual motion stimuli.

  2. Our PSE as percent of deviation from standard IOI is the same index as the relative constant error (CE) in the studies of Miller and McAuley (2005) and McAuley and Miller (2007).

  3. When a mean ITI exceeded three standard deviations from the average of mean ITIs under the same condition, the trial was excluded from the analysis. The occurrence of exclusion was generally very low, never exceeding 2% in any participant.

References

  • Alais D, Burr D (2004) The ventriloquist effect results from near-optimal bimodal integration. Curr Biol 14:257–262

    PubMed  CAS  Google Scholar 

  • Alink A, Singer W, Muckli L (2008) Capture of auditory motion by vision is represented by an activation shift from auditory to visual motion cortex. J Neurosci 28:2690–2697

    Article  PubMed  CAS  Google Scholar 

  • Bengtsson SL, Ullen F, Ehrsson HH, Hashimoto T, Kito T, Naito E, Forssberg H, Sadato N (2009) Listening to rhythms activates motor and premotor cortices. Cortex 45:62–71

    Article  PubMed  Google Scholar 

  • Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433–436

    Article  PubMed  CAS  Google Scholar 

  • Brandon M, Saffran JR (2011) Apparent motion enhances visual rhythm discrimination in infancy. Atten Percept Psychophys. doi:10.3758/s13414-011-0106-x

  • Buhusi C, Meck WH (2005) What makes us tick? Functional and neural mechanisms of interval timing. Nat Rev Neurosci 6:755–765

    Article  PubMed  CAS  Google Scholar 

  • Burr D, Banks M, Morrone M (2009) Auditory dominance over vision in the perception of interval duration. Exp Brain Res 198:49–57

    Article  PubMed  Google Scholar 

  • Chapin HL, Zanto T, Jantzen KJ, Kelso SJA, Steinberg F, Large EW (2010) Neural response to complex auditory rhythms: the role of attending. Front Psychol 1:1–18

    Article  Google Scholar 

  • Chen JL, Penhune VB, Zatorre RJ (2008) Listening to musical rhythms recruits motor regions of the brain. Cereb Cortex 18:2844–2854

    Article  PubMed  Google Scholar 

  • Chen JL, Penhune VB, Zatorre RJ (2009) The role of auditory and premotor cortex in sensorimotor transformations. Ann NY Acad Sci 1169:15–34

    Article  PubMed  Google Scholar 

  • Drake C, Botte MC (1993) Tempo sensitivity in auditory sequences: evidence for a multi-look model. Percept Psychophys 54:277–286

    Article  PubMed  CAS  Google Scholar 

  • Drake C, Jones MR, Baruch C (2000) The development of rhythmic attending in auditory sequences: attunement, referent period, focal attending. Cognition 77:251–288

    Article  PubMed  CAS  Google Scholar 

  • Droit-Volet S, Wearden JH (2002) Speeding up an internal clock in children? Effects of visual flicker on subjective duration. Q J Exp Psychol B 55:193–211

    Article  PubMed  Google Scholar 

  • Fadiga L, Fogassi L, Gallese V, Rizzolatti G (2000) Visuomotor neurons: ambiguity of the discharge or ‘motor’ perception? Int J Psychophysiol 35:165–177

    Article  PubMed  CAS  Google Scholar 

  • Fendrich R, Corballis PM (2001) The temporal cross-capture of audition and vision. Percept Psychophys 63:719–725

    Article  PubMed  CAS  Google Scholar 

  • Fraisse P (1963) Psychology of time. Harper & Row, New York

    Google Scholar 

  • Fries P, Reynolds JH, Rorie AE, Desimone R (2001) Modulation of oscillatory neuronal synchronization by selective visual attention. Science 291:1560–1563

    Google Scholar 

  • Grahn JA, Brett M (2007) Rhythm and beat perception in motor areas of the brain. J Cogn Neurosci 19:893–906

    Article  PubMed  Google Scholar 

  • Grahn JA, Rowe JB (2009) Feeling the beat: premotor and striatal interactions in musicians and nonmusicians during beat perception. J Neurosci 29:7540–7548

    Article  PubMed  CAS  Google Scholar 

  • Grahn JA, Henry MJ, McAuley JD (2010) FMRI investigation of cross-modal interactions in beat perception: audition primes vision, but not vice versa. Neuroimage. doi:10.1016/j.neuroimage.2010.09.033

  • Grondin S, McAuley D (2009) Duration discrimination in crossmodal sequences. Perception 38:1542–1559

    Article  PubMed  Google Scholar 

  • Guttman SE, Gilroy LA, Blake R (2005) Hearing what the eyes see: auditory encoding of visual temporal sequences. Psychol Sci 16:228–265

    Article  PubMed  Google Scholar 

  • Hoshi E, Tanji J (2006) Differential involvement of neurons in the dorsal and ventral premotor cortex during processing of visual signals for action planning. J Neurophysiol 95:3596–3616

    Article  PubMed  Google Scholar 

  • Hove MJ, Spivey MJ, Krumhansl CL (2010) Compatibility of motion facilitates visuomotor synchronization. J Exp Psychol Hum Percept Perform 36:1525–1534

    Article  PubMed  Google Scholar 

  • Iversen JR, Repp BH, Patel AD (2009) Top-down control of rhythm perception modulates early auditory responses. Ann NY Acad Sci 1169:58–73

    Article  PubMed  Google Scholar 

  • Jones MR, Boltz M (1989) Dynamic attending and responses to time. Psychol Rev 96:459–491

    Article  PubMed  CAS  Google Scholar 

  • Jones MR, McAuley JD (2005) Time judgments in global temporal contexts. Percept Psychophys 67:398–417

    Article  PubMed  Google Scholar 

  • Kanai R, Paffen CL, Gerbino W, Verstraten FA (2004) Blindness to inconsistent local signals in motion transparency from oscillating dots. Vision Res 44:2207–2212

    PubMed  Google Scholar 

  • Kanai R, Paffen CLE, Hogendoorn H, Verstraten FAJ (2006) Time dilation in dynamic visual display. J Vis 9:1421–1430. doi:10.1167/6.12.8

    Google Scholar 

  • Kaneko S, Murakami I (2009) Perceived duration of visual motion increases with speed. J Vis 9:1–12. doi:10.1167/9.7.14

    Article  Google Scholar 

  • Karabanov A, Blom Ö, Forsman L, Ullén F (2009) The dorsal auditory pathway is involved in performance of both visual and auditory rhythms. Neuroimage 44:480–488

    Article  PubMed  Google Scholar 

  • Kitagawa N, Ichihara S (2002) Hearing visual motion in depth. Nature 416:172–174

    Article  PubMed  CAS  Google Scholar 

  • Kohler E, Keysers C, Umiltà MA, Fogassi L, Gallese V, Rizzolatti G (2002) Hearing sounds, understanding actions: action representation in mirror neurons. Science 297:846–848

    Article  PubMed  CAS  Google Scholar 

  • Kornysheva K, von Cramon DY, Jacobsen T, Schubotz RI (2010) Tuning-into the beat: aesthetic appreciation of musical rhythms correlates with a premotor activity boost. Hum Brain Mapp 31:48–64

    PubMed  Google Scholar 

  • Kovács G, Raabe M, Greenlee MW (2008) Neural correlates of visually induced self-motion illusion in depth. Cereb Cortex 18:1779–1787

    Article  PubMed  Google Scholar 

  • Lahav A, Saltzman E, Schlaug G (2007) Action representation of sound: audiomotor recognition network while listening to newly acquired actions. J Neurosci 27:308–314

    Article  PubMed  CAS  Google Scholar 

  • Lakatos P, Karmos G, Mehta AD, Ulbert I, Schroeder CE (2008) Entrainment of neuronal oscillations as a mechanism of attentional selection. Science 320:110–113

    Article  PubMed  CAS  Google Scholar 

  • Large EW (2008) Resonating to musical rhythm: theory and experiment. In: Grondin S (ed) The psychology of time. Emerald, UK, pp 189–231

    Google Scholar 

  • Large EW, Snyder JS (2009) Pulse and meter as neural resonance. Ann NY Acad Sci 1169:46–57

    Article  PubMed  Google Scholar 

  • Lee J, van Donkelaar P (2006) The human dorsal premotor cortex generates on-line error corrections during sensorimotor adaptation. J Neurosci 26:3330–3334

    Article  PubMed  CAS  Google Scholar 

  • Lewald J, Guski R (2003) Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. Cogn Brain Res 16:468–478

    Article  Google Scholar 

  • Matthews WJ (2011) How do changes in speed affect the perception of duration? J Exp Psychol Human. doi:10.1037/a0022193

  • McAuley JD, Henry MJ (2010) Modality effects in rhythm processing: auditory encoding of visual rhythms is neither obligatory nor automatic. Atten Percept Psychophys 72:1377–1389

    Article  PubMed  Google Scholar 

  • McAuley JD, Jones MR (2003) Modeling effects of rhythmic context on perceived duration: comparison of interval and entrainment approaches to short-interval timing. J Exp Psychol Hum 29:1102–1125

    Article  Google Scholar 

  • McAuley JD, Miller NS (2007) Effects of global temporal context on sensitivity to the tempo of auditory sequences. Percept Psychophys 69:709–718

    Article  PubMed  Google Scholar 

  • Merker BH, Madison GS, Eckerdal P (2009) On the role and origin of isochrony in human rhythmic entrainment. Cortex 45:4–17

    Article  PubMed  Google Scholar 

  • Miller NS, McAuley JD (2005) Tempo sensitivity in isochronous tone sequences: the multiple-look model revisited. Percept Psychophys 67:1150–1160

    Article  PubMed  Google Scholar 

  • Morein-Zamir S, Soto-Faraco S, Kingston A (2003) Auditory capture of vision: examining temporal ventriloquism. Cogn Brain Res 17:154–163

    Article  Google Scholar 

  • Murata A, Fadiga L, Fogassi L, Gallese V, Raos V, Rizzolatti G (1997) Object representation in the ventral premotor cortex (area F5) of the monkey. J Neurophysiol 78:2226–2230

    PubMed  CAS  Google Scholar 

  • Patel AD, Iversen JR, Chen Y, Repp BH (2005) The influence of metricality and modality on synchronization with a beat. Exp Brain Res 163:226–238

    Article  PubMed  Google Scholar 

  • Pilgramm S, Lorey B, Stark R, Munzert J, Vaitl D, Zentgraf K (2010) Differential activation of the lateral premotor cortex during action observation. BMC Neurosci 11:89. doi:10.1186/1471-2202-11-89

    Article  PubMed  Google Scholar 

  • Prinz W (1997) Perception and action planning. Eur J Cogn Psychol 9:129–154

    Article  Google Scholar 

  • Recanzone GH (2003) Auditory influences on visual temporal rate perception. J Neurophysiol 89:1078–1093

    Article  PubMed  Google Scholar 

  • Repp BH (2003) Rate limits in sensorimotor synchronization with auditory and visual sequences: the synchronization threshold and the benefits and costs of interval subdivision. J Mot Behav 35:355–370

    Article  PubMed  Google Scholar 

  • Repp BH (2008) Metrical subdivision results in subjective slowing of the beat. Music Percept 26:19–39

    Article  Google Scholar 

  • Repp BH, Bruttomesso M (2009) A filled duration illusion in music: effects of metrical subdivision on the perception and production of beat tempo. Adv Cogn Psychol 5:114–134

    Article  Google Scholar 

  • Repp BH, Penel A (2002) Auditory dominance in temporal processing: new evidence from synchronization with simultaneous visual and auditory sequences. J Exp Psychol Hum Percept Perform 28:1085–1099

    Article  PubMed  Google Scholar 

  • Rizzolatti G, Fadiga L, Gallesi V, Fogassi L (1996) Premotor cortex and the recognition of motor actions. Brain Res Cogn Brain Res 3:131–141

    Article  PubMed  CAS  Google Scholar 

  • Scheef L, Boecker H, Daamen M, Fehse U, Landsberg MW, Granath DO, Mechling H, Effenberg AO (2009) Multimodal motion processing in area V5/MT: evidence from an artificial class of audio-visual events. Brain Res 1252:94–104

    Article  PubMed  CAS  Google Scholar 

  • Schmidt RC, Richardson MJ, Arsenault C, Galantucci B (2007) Visual tracking and entrainment to an environmental rhythm. J Exp Psychol Human 33:860–870

    Article  CAS  Google Scholar 

  • Shams L, Kamitani Y, Shimojo S (2000) Illusions. What you see is what you hear. Nature 408:788

    Article  PubMed  CAS  Google Scholar 

  • Shams L, Ma WJ, Beierholm U (2005) Sound-induced flash illusion as an optimal percept. Neuroreport 16:1923–1927

    Article  PubMed  Google Scholar 

  • Shipley T (1964) Auditory flutter-driving of visual flicker. Science 145:1328–1330

    Article  PubMed  CAS  Google Scholar 

  • Soto-Faraco S, Lyons J, Gazzaniga M, Spence C, Kingston A (2002) The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. Cogn Brain Res 14:139–146

    Article  Google Scholar 

  • Todd NPM (1999) Motion in music: a neurobiological perspective. Music Percept 17:115–126

    Google Scholar 

  • Todd NPM, Lee CS, O’Boyle DJ (2002) A sensorimotor theory of temporal tracking and beat induction. Psychol Res 66:26–39

    Article  PubMed  Google Scholar 

  • Treisman M, Faulkner A, Naish PLN, Brogan D (1990) The internal clock: evidence for a temporal oscillator underlying time perception with some estimates of its characteristic frequency. Perception 19:705–743

    Article  PubMed  CAS  Google Scholar 

  • Tse PU, Intriligator J, Rivest J, Cavanagh P (2004) Attention and the subjective expansion of time. Percept Psychophys 66:1171–1189

    Article  PubMed  Google Scholar 

  • van der Hoorn A, Beudel M, de Jong BM (2010) Interruption of visually perceived forward motion in depth evokes a cortical activation shift from spatial to intentional motor regions. Brain Res 1358:160–171

    Article  PubMed  Google Scholar 

  • van Wassenhove V, Buonomano D, Shimojo S, Shams L (2008) Distortions of subjective time perception within and across senses. Plos One 3:e1437. doi:10.1371/journal.pone.0001437

    Article  PubMed  Google Scholar 

  • Vierordt K (1868) Der Zeitsinn nach Versuchen. Tübingen, Laupp

    Google Scholar 

  • Warren DH, Welch RB, McCarthy TJ (1981) The role of visual-auditory “compellingness” in the ventriloquism effect: implications for transitivity among the spatial senses. Percept Psychophys 30:557–564

    Article  PubMed  CAS  Google Scholar 

  • Welch RB, Warren DH (1980) Immediate perceptual response to intersensory discrepancy. Psychol Bull 88:638–667

    Article  PubMed  CAS  Google Scholar 

  • Wittmann M, van Wassenhove V, Craig AD, Paulus MP (2010) The neural substrates of subjective time dilation. Front Hum Neurosci 4. doi:10.3389/neuro.09.002.2010

Download references

Acknowledgments

This work was supported by a doctoral scholarship to the first author from the Bayerische Forschungsstiftung, and the experimental expenses were additionally supported by the Andrea von Braun Stiftung. The authors thank Bruno Repp for very helpful comments on earlier versions of the manuscript.

Author information

Authors and Affiliations

  1. Institute of Medical Psychology, Ludwig-Maximilians-University, Goethestr. 31, 80336, Munich, Germany

    Yi-Huang Su

  2. Allgemeine und Experimentelle Psychologie, Ludwig-Maximilians-University, Munich, Germany

    Donatas Jonikaitis

Authors
  1. Yi-Huang Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Donatas Jonikaitis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi-Huang Su.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Su, YH., Jonikaitis, D. Hearing the speed: visual motion biases the perception of auditory tempo. Exp Brain Res 214, 357–371 (2011). https://doi.org/10.1007/s00221-011-2835-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-011-2835-4

Keywords

Search

Navigation