Skip to main content
SpringerLink
Log in

Multisensory integration of drumming actions: musical expertise affects perceived audiovisual asynchrony

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

We investigated the effect of musical expertise on sensitivity to asynchrony for drumming point-light displays, which varied in their physical characteristics (Experiment 1) or in their degree of audiovisual congruency (Experiment 2). In Experiment 1, 21 repetitions of three tempos × three accents × nine audiovisual delays were presented to four jazz drummers and four novices. In Experiment 2, ten repetitions of two audiovisual incongruency conditions × nine audiovisual delays were presented to 13 drummers and 13 novices. Participants gave forced-choice judgments of audiovisual synchrony. The results of Experiment 1 show an enhancement in experts’ ability to detect asynchrony, especially for slower drumming tempos. In Experiment 2 an increase in sensitivity to asynchrony was found for incongruent stimuli; this increase, however, is attributable only to the novice group. Altogether the results indicated that through musical practice we learn to ignore variations in stimulus characteristics that otherwise would affect our multisensory integration processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Using a series of static frames sampled at 30 Hz like Arrighi et al. (2006) would increase the possibility that the frame showing the actual point of contact between the percussor and the resonator is not actually shown (see Arrighi et al. 2006, p 266); we used QuickTime movies sampled at 60 Hz to maintain as much as possible the original information of the drummer’s performance.

  2. The estimate of TIW width was derived by calculating the SD of the normal distribution, which is known to be equal to FWHM/2.3548. The FWHM or full width at half maximum is a simple measure of the width of a distribution, and is easily obtained from empirical distributions. The FWHM for a distribution described by the probability density f(x) is defined by the absolute difference between one point on the left x 2 and one on the right x 1 of the mode x m (defined by f(x m ) = max), with f(x 1) = f(x 2) = f(x m )/2.

  3. R² measures how successful the fit is in explaining the variation of the data. That is, R² is the square of the correlation between the response values and the predicted response values. R² is defined as the ratio of the sum of squares of the regression (SSR) and the total sum of squares (SST). R² can take on any value between 0 and 1, with a value closer to 1 indicating that a greater proportion of variance is accounted for by the model.

References

  • Arai T, Greenberg S (1997). The temporal properties of spoken Japanese are similar to those of English. In: Proceedings of Eurospeech, Rhodes, Greece, pp 1011–1014

  • Arrighi R, Alais D, Burr D (2006) Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. J Vision 6:260–268

    Article  Google Scholar 

  • Bengtsson SL, Nagy Z, Skare S, Forsman L, Forssberg H, Ullén F (2005) Extensive piano practicing has regionally specific effects on white matter development. Nature Neurosci 8:1148–1150

    Article  PubMed  CAS  Google Scholar 

  • Bermudez P, Zatorre RJ (2005) Differences in gray matter between musicians and nonmusicians. Ann N Y Acad Sci 1060:395–399

    Article  PubMed  Google Scholar 

  • Brainard DH (1997) The psychophysics toolbox. Spatial Vision 10:433–436

    Article  PubMed  CAS  Google Scholar 

  • Brooks A, van der Zwan R, Billard A, Petreska B, Clarke S, Blanke O (2007) Auditory motion affects visual biological motion processing. Neuropsychologia 45:523–530

    Article  PubMed  CAS  Google Scholar 

  • Clifford CWG, Arnold DH, Pearson J (2003) A paradox of temporal perception revealed by a stimulus oscillating in colour and orientation. Vision Res 43:2245–2253

    Article  PubMed  Google Scholar 

  • Collyer CE, Broadbent HA, Church RM (1994) Preferred rates of repetitive tapping and categorical time production. Percept Psychophys 55:443–453

    PubMed  CAS  Google Scholar 

  • Dahl S (2004) Playing the accent—comparing striking velocity and timing in an ostinato rhythm performed by four drummers. Acta Acustica United Acustica 90:762–776

    Google Scholar 

  • Dixon NF, Spitz L (1980) The detection of auditory visual desynchrony. Perception 9:719–721

    Article  PubMed  CAS  Google Scholar 

  • Drake C, Botte MC (1993) Tempo sensitivity in auditory sequences: evidence for a multiple-look model. Percept Psychophys 54:277–286

    PubMed  CAS  Google Scholar 

  • Drake C, Jones MR, Baruch C (2000) The development of rhythmic attending in auditory sequences: attunement, referent period, focal attending. Cognition 77:251–288

    Article  PubMed  CAS  Google Scholar 

  • Fain GL (2003) Sensory transduction. Sinauer Associates, Sunderland, MA

    Google Scholar 

  • Fontana F, Rocchesso D, Avanzini F (2004) Computation of nonlinear filter networks containing delay-free paths. In Proceedings of the 7th international conference on digital audio effects (DAFX-04), Naples, Italy, pp 113–118

  • Friberg A, Sundberg J (1995) Time discrimination in a monotonic, isochronous sequence. J Acoust Soc Am 98:2524–2531

    Article  Google Scholar 

  • Fujisaki W, Nishida S (2005) Temporal frequency characteristics of synchrony–asynchrony discrimination of audio–visual signals. Exp Brain Res 166:455–464

    Article  PubMed  Google Scholar 

  • Gaser C, Schlaug G (2003) Brain structures differ between musicians and non-musicians. J Neurosci 23:9240–9245

    PubMed  CAS  Google Scholar 

  • Grant KW, Greenberg S (2001) Speech intelligibility derived from asynchronous processing of auditory–visual speech information. In: Proceedings of AVSP 2001 international conference of auditory–visual speech processing, Scheelsminde, Denmark, pp 132–137

  • Grant KW, van Wassenhove V, Poeppel D (2004) Detection of auditory (cross-spectral) and auditory–visual (cross-modal) synchrony. J Acoust Soc Am 108:1197–1208

    Article  Google Scholar 

  • Hodges DA, Hairston WD, Burdette JH (2005) Aspects of multisensory perception: the integration of visual and auditory information in musical experiences. Ann N Y Acad Sci 1060:175–185

    Article  PubMed  Google Scholar 

  • Hollier MP, Rimell AN (1998) An experimental investigation into multi-modal synchronisation sensitivity for perceptual model development. 105th AES Convention Preprint No. 4790

  • Johansson G (1973) Visual perception of biological motion and model for its analysis. Percept Psychophys 14:201–211

    Google Scholar 

  • Jones MR (1987) Dynamic pattern structure in music: recent theory and research. Percept Psychophys 41:621–634

    PubMed  CAS  Google Scholar 

  • Kay BA, Kelso JAS, Saltzman EL, Schöner G (1987) Space–time behavior of single and bimanual rhythmical movements: data and limit cycle model. J Exp Psychol: Human Percept Perform 13:178–192

    Article  CAS  Google Scholar 

  • King AJ (2005) Multisensory integration: strategies for synchronization. Curr Biol 15:R339–R341

    Article  PubMed  CAS  Google Scholar 

  • King AJ, Palmer AR (1985) Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Exp Brain Res 60:492–500

    Article  PubMed  CAS  Google Scholar 

  • Luck G, Nte S (2008) An investigation of conductors’ temporal gestures and conductor— musician synchronization, and a first experiment. Psychol Music 36:81–99

    Article  Google Scholar 

  • Luck G, Sloboda JA (2007) An investigation of musicians’ synchronization with traditional conducting beat patterns. Music Perform Res 1:26–46

    Google Scholar 

  • MacDonald J, McGurk H (1978) Visual influences on speech perception processes. Percept Psychophys 24:253–257

    PubMed  CAS  Google Scholar 

  • MacDougall HG, Moore ST (2005) Marching to the beat of the same drummer: the spontaneous tempo of human locomotion. J Appl Physiol 99:1164–1173

    Article  PubMed  Google Scholar 

  • Massaro DW, Cohen MM, Smeele PM (1996) Perception of asynchronous and conflicting visual and auditory speech. J Acoust Soc Am 100:1777–1786

    Article  PubMed  CAS  Google Scholar 

  • McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748

    Article  PubMed  CAS  Google Scholar 

  • Miner N, Caudell T (1998) Computational requirements and synchronization issues for virtual acoustic displays. Presence-Teleoperat Virtual Environ 7(4):396–409

    Article  Google Scholar 

  • Moelants D (2002) Preferred tempo reconsidered. In: Proceedings of the 7th international conference on music perception and cognition, Sydney, 2002, Stevens C, Burnham D, McPherson G, Schubert E, Renwick J (eds). Causal productions, Adelaide, pp 580–583

  • Munhall KG, Gribble P, Sacco L, Ward M (1996) Temporal constraints on the McGurk effect. Percept Psychophys 58:351–362

    PubMed  CAS  Google Scholar 

  • Murray M, Drought AB, Kory RC (1964) Walking patterns of normal men. J Bone Joint Surgery 46:335–360

    CAS  Google Scholar 

  • Navarra J, Vatakis A, Zampini M, Soto-Faraco S, Humphreys W, Spence C (2005) Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Cognitive Brain Res 25(2):499–507

    Article  Google Scholar 

  • Palmer C, Krumhansl CL (1990) Mental representations for musical meter. J Exp Psychol Hum Percept Perform 16:728–741

    Article  PubMed  CAS  Google Scholar 

  • Pelli DG (1997) The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision 10:437–442

    Article  PubMed  CAS  Google Scholar 

  • Petrini K, Russell M, Pollick F (2009) When knowing can replace seeing in audiovisual integration of actions. Cognition 110:432–439

    Article  PubMed  Google Scholar 

  • Saygin AP, Driver J, de Sa VR (2008) In the footsteps of biological motion and multisensory perception: judgements of audio-visual temporal relations are enhanced for upright walkers. Psychol Sci 19:469–475

    Article  PubMed  Google Scholar 

  • Schutz M, Kubovy M (2008) The effect of tone envelope on sensory integration: support for the ‘unity assumption’. J Acoust Soc Am 123:3412

    Article  Google Scholar 

  • Schutz M, Lipscomb S (2007) Hearing gestures, seeing music: vision influences perceived tone duration. Perception 36:888–897

    Article  PubMed  Google Scholar 

  • Spence C, Squire S (2003) Multisensory integration: maintaining the perception of synchrony. Curr Biol 13:519–521

    Article  CAS  Google Scholar 

  • Spence C, Walton M (2005) On the inability to ignore touch when responding to vision in the crossmodal congruency task. Acta Psychol 118:47–70

    Article  Google Scholar 

  • Spence C, Shore DI, Klein RM (2001) Multisensory prior entry. J Exp Psychol: Gen 130:799–832

    Article  CAS  Google Scholar 

  • Steinmetz R (1996) Human perception of jitter and media synchronization. IEEE 14:61–72

    Google Scholar 

  • Stone JV, Hunkin NM, Porrill J, Wood R, Keeler V, Beanland M, Port M, Porter NR (2001) When is now? Perception of simultaneity. Proc R Soc, B 268:31–38

    Article  CAS  Google Scholar 

  • Sugita Y, Suzuki Y (2003) Audiovisual perception: implicit estimation of sound-arrival time. Nature 421:911

    Article  PubMed  CAS  Google Scholar 

  • van Wassenhove V, Grant KW, Poeppel D (2007) Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45:598–607

    Article  PubMed  Google Scholar 

  • Vatakis A, Spence C (2006a) Audiovisual synchrony perception for speech and music using a temporal order judgment task. Neurosci Lett 393:40–44

    Article  PubMed  CAS  Google Scholar 

  • Vatakis A, Spence C (2006b) Audiovisual synchrony perception for music, speech, and object actions. Brain Res 1111:134–142

    Article  PubMed  CAS  Google Scholar 

  • Vatakis A, Spence C (2007) Crossmodal binding: evaluating the “unity assumption” using audiovisual speech stimuli. Percept Psychophys 69:744–756

    PubMed  Google Scholar 

  • Vatakis A, Spence C (2008) Evaluating the influence of the ‘unity assumption’ on the temporal perception of realistic audiovisual stimuli. Acta Psychol 127:12–23

    Article  Google Scholar 

  • Vatakis A, Navarra J, Soto-Faraco S, Spence C (2007a) Audiovisual temporal adaptation of speech: temporal order versus simultaneity judgments. Exp Brain Res 185:521–529

    Article  PubMed  Google Scholar 

  • Vatakis A, Navarra J, Soto-Faraco S, Spence C (2007b) Temporal recalibration during asynchronous audiovisual speech perception. Exp Brain Res 181:173–181

    Article  PubMed  Google Scholar 

  • Vatakis A, Ghazanfar AA, Spence C (2008) Facilitation of multisensory integration by the “unity effect” reveals that speech is special. J Vision 8:1–11

    Article  Google Scholar 

  • Vroomen J, Keetels M, de Gelder B, Bertelson P (2004) Recalibration of temporal order perception by exposure to audio-visual asynchrony. Cognitive Brain Res 22:32–35

    Article  Google Scholar 

  • Waadeland CH (2003) Analysis of jazz drummers’ movements in performance of swing grooves-a preliminary report. In: Bresin R (ed) Proceedings of Stockholm music acoustic conference, volume II, Stockholm, pp 573–576

  • Waadeland CH (2006) Strategies in empirical studies of swing groove. Studia Musicol Norvegica 32:169–191

    Google Scholar 

  • Wagner A (2006) Analysis of drumbeats—interaction between Drummer, Drumstick and Instrument. Master’s Thesis at the Department of Speech, Music and Hearing (TMH)

  • Watson AB, Hu J (1999) ShowTime: a QuickTime-based infrastructure for vision research displays. Perception 28 ECVP Abstract Supplement, 45b

  • Zampini M, Shore DI, Spence C (2003) Audiovisual temporal order judgments. Exp Brain Res 152:198–210

    Article  PubMed  Google Scholar 

  • Zhou F, Wong V, Sekuler R (2007) Multi-sensory integration of spatio-temporal segmentation cues: one plus one does not always equal two. Exp Brain Res 180:641–654

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This work was supported by a grant from the British Academy (LRG-42455). We would like to thank Melanie Russell for her contribution during the pilot phase of this study and Gerry Rossi of Strathclyde University for his help in recruiting the jazz drummers. We also would like to thank Jim Kay and Michael Kubovy for their valuable suggestions on data analysis and statistical methods.

Author information

Authors and Affiliations

  1. Department of Psychology, University of Glasgow, 58 Hillhead Street, Glasgow, G12 8QB, Scotland, UK

    Karin Petrini & Frank E. Pollick

  2. Department of Media Technology, Aalborg University Copenhagen, Copenhagen, Denmark

    Sofia Dahl

  3. Department of Art and Industrial Design, IUAV University of Venice, Venice, Italy

    Davide Rocchesso

  4. Department of Music, Norwegian University of Science and Technology, Trondheim, Norway

    Carl Haakon Waadeland

  5. Department of Information Engineering, University of Padova, Padova, Italy

    Federico Avanzini

  6. Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA

    Aina Puce

Authors
  1. Karin Petrini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sofia Dahl
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Davide Rocchesso
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carl Haakon Waadeland
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Federico Avanzini
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Aina Puce
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Frank E. Pollick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karin Petrini.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Petrini, K., Dahl, S., Rocchesso, D. et al. Multisensory integration of drumming actions: musical expertise affects perceived audiovisual asynchrony. Exp Brain Res 198, 339–352 (2009). https://doi.org/10.1007/s00221-009-1817-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-009-1817-2

Keywords

Search

Navigation