Elsevier

Human Movement Science

Volume 32, Issue 1, February 2013, Pages 21-47
Human Movement Science

Small perturbations in a finger-tapping task reveal inherent nonlinearities of the underlying error correction mechanism

https://doi.org/10.1016/j.humov.2012.06.002Get rights and content

Abstract

Time processing in the few hundred milliseconds range is involved in the human skill of sensorimotor synchronization, like playing music in an ensemble or finger tapping to an external beat. In finger tapping, a mechanistic explanation in biologically plausible terms of how the brain achieves synchronization is still missing despite considerable research. In this work we show that nonlinear effects are important for the recovery of synchronization following a perturbation (a step change in stimulus period), even for perturbation magnitudes smaller than 10% of the period, which is well below the amount of perturbation needed to evoke other nonlinear effects like saturation. We build a nonlinear mathematical model for the error correction mechanism and test its predictions, and further propose a framework that allows us to unify the description of the three common types of perturbations. While previous authors have used two different model mechanisms for fitting different perturbation types, or have fitted different parameter value sets for different perturbation magnitudes, we propose the first unified description of the behavior following all perturbation types and magnitudes as the dynamical response of a compound model with fixed terms and a single set of parameter values.

Highlights

► We applied Nonlinear Dynamics to finger-tapping data and unveiled nontrivial features. ► Nonlinear features of the behavior are important even at small perturbation magnitudes. ► Few studies consistently include the perturbation within the modeling effort. ► We propose a simple mathematical model that correctly predicts nontrivial behavior. ► A single set of parameter values accounts for all perturbation types and magnitudes.

Introduction

Time perception and production in the range of several hundreds of milliseconds, known as millisecond timing, is crucial for motor control, speech generation and recognition, playing music and dancing, and rapid sequencing of cognitive operations such as updating working memory (Buhusi and Meck, 2005, Meck, 2005). However, our understanding of the basic mechanisms underlying these behaviors is poor, and the representation of temporal information in the brain remains one of the most elusive concepts in neurobiology (Ivry & Spencer, 2004), particularly in this timing range. To date no strong consensus has been reached about which brain regions are involved in time measurement of short intervals and how they interact (Beudel et al., 2009, Del Olmo et al., 2008, Lewis and Miall, 2003, Manto and Bastian, 2007), or which is the neural mechanism responsible for the production of timed responses in this range (Buonomano & Laje, 2010). This stands in stark contrast to our rather comprehensive knowledge about temporal processing in other ranges, for example circadian timing (Panda, Hogenesch, & Kay, 2002).

A paradigmatic aspect of millisecond timing is sensorimotor synchronization, which is the ability to entrain movement to an external metronome. Although a recent study reported this ability in a variety of non-human species (Schachner, Brady, Pepperberg, & Hauser, 2009), animals display a very limited form of the behavior. In contrast, it is quite easy for humans to achieve synchronization with a metronome or musical beat and this forms the basis of all music and dance. One of the simplest tasks to study sensorimotor synchronization is finger tapping. In this task a subject is instructed to tap in synchrony with a periodic sequence of brief tones, and the time difference between each response and its corresponding stimulus is recorded (see Fig. 1). Despite its simplicity, this task helps to unveil interesting features of the underlying neural system and the error correction mechanism responsible for synchronization.

The first evidence of the existence of such a correction mechanism is the phenomenon of synchronization itself; although no single response is perfectly aligned in time with the corresponding stimulus, the responses stay in the vicinity of the corresponding stimuli throughout (see Fig. 1; note the commonly observed tendency of anticipation, called Negative Mean Asynchrony or NMA). Without a correction mechanism, tiny synchronization errors or small differences between the interstimulus interval and the interresponse interval would rapidly accumulate and make the responses drift away from the stimuli, as it is very unlikely that the subject could set his/her interresponse interval exactly at the right value—even on average. This is most evident when the subject is instructed to keep tapping at the same pace after the metronome has been muted, what is called a continuation paradigm. The “virtual asynchronies” computed between the continuing taps and the extrapolated silent beats usually get quite large within a few taps (Repp, 2005), even for musically trained subjects. Note that this crude evidence for a correction mechanism does not indicate the kind of mechanism used, since average synchronization could be achieved through either continuous adjustments (i.e., at every step), or intermittent control (i.e., once every few steps), or some other correction strategy (Gross et al., 2002), or even a mix of short- and long-range processes (Wagenmakers, Farrell, & Ratcliff, 2004).

The behavior of the neural mechanisms underlying finger tapping synchronization is usually interpreted in terms of an error-correction function f that takes past events as inputs (including asynchronies, intervals, and interval differences) and estimates the timing of the next response. This approach assumes that the underlying mechanism can be separated into a deterministic part (the correction function itself) and noise (due to inherent variability of time estimation, motor action, etc., see the seminal paper on clock and motor variance by Wing & Kristofferson, 1973Wing & Kristofferson (1973)). The form of the error correction function can then be generally stated as:en+1=f(en,tn,rn,Tn,)+noisewhere en is the synchrony error at step n, Tn is the interstimulus interval, and rn is the interresponse interval; the function f could have a dependency on earlier steps as well (n  1, n  2, etc.). The variable tn is usually associated with the period of a presumed internal timekeeper (Wing & Kristofferson, 1973), as in Mates’ influential dual-process error correction model for sensorimotor synchronization (Mates, 1994a, Mates, 1994b). In the dual-process model the subject is capable of adjusting the phase and period of the internal timekeeper. The error correction function is usually chosen to be linear, but nonlinear terms may be needed to accommodate saturation effects observed in perturbation experiments with large perturbation magnitudes up to 50% of the stimulus period (Repp, 2002b) or following spontaneous large synchrony errors (Engbert, Krampe, Kurths, & Kliegl, 2002). The noise term in Eq. (1) may have a complicated structure of its own (e.g., Vorberg & Schulze, 2002), and experimental data obtained with isochronous sequences can display long-range correlations (known as 1/f noise; Chen et al., 1997, Wagenmakers et al., 2004).

Related work within the internal timekeeper framework can be traced as far back as the pioneering work by Michon (1967), who proposed a linear predictive model that estimates the next interresponse interval based on the preceding two interstimulus intervals. Hary and Moore (1987a) proposed the influential hypothesis that the subject estimates the timing of the next tap based randomly on either the preceding stimulus occurrence Sn or the preceding tap Rn, a strategy called “mixed phase resetting” that was later shown to be equivalent to the (now usual) assumption that the estimation is based on the synchrony error en (Schulze, 1992). It is still under debate which perceptual information is used by the error correction mechanism; Schulze, Cordes, and Vorberg (2005) proposed an alternative to Mates’ model in which the internal timekeeper period is updated by the preceding synchrony error en, instead of the difference between the preceding interstimulus interval and timekeeper period, as Mates (1994a) has proposed. Even considering only one source of perceptual information, Pressing and Jolley-Rogers (1997) showed that the subject’s response to an isochronous sequence can depend on the last error en or the last two errors en and en−1, depending on whether the sequence is slow or fast, and accordingly proposed a second-order autoregressive model (Pressing, 1998, Pressing and Jolley-Rogers, 1997).

An alternative theoretical framework to explain the coordination of a subject’s response with a periodic sequence is based on the concepts of self-sustained attentional oscillation, phase entrainment, and period adaptation (Large, 2000, Large and Jones, 1999). It assumes that the external rhythmic signal evokes intrinsic neural attentional oscillations that entrain to the periodicities of the sequence (Loehr, Large, & Palmer, 2011), a process that can be represented by a sine circle map—a nonlinear system that can show more complex entrainments than 1:1 synchronization, such as 2:1, etc. Although oscillator and timekeeper models seem very different at first sight, linear timekeeper models can be seen as simplifications of nonlinear oscillator models and actually yield similar predictions for certain types of behavior (Loehr et al., 2011). Oscillator models probably surpass timekeeper models when stimuli with multiple periodicities are considered (Loehr et al., 2011); to the best of our knowledge, however, the extent to which they accurately reproduce the transient behavior when abrupt perturbations are considered has yet to be demonstrated—the perturbations considered by Loehr et al. (2011) were not abrupt step changes but slow linear increases or decreases in tempo, which in fact can be seen as quasi-stationary if the model is in a high dissipation regime.

There is evidence of nonlinearity in finger tapping tasks. Repp, 2002b, Repp, 2011a showed that the response to a phase-shift perturbation displays at least two distinctive nonlinear features: asymmetry and saturation in the phase correction response (PCR, equal to the difference between the time of occurrence of the first tap after the perturbation and the expected time when this tap would have occurred in the absence of perturbation). Asymmetry was evident as smaller values of the PCR for negative than for positive shifts, but only for perturbation magnitudes greater than ±10% of a 500 ms period (±50 ms). Saturation effects, displayed as a shallower increase of the PCR function with the size of large perturbations than of small perturbations, were also evident. Interestingly, Repp (2002b) also found an asymmetry in the standard deviation of the PCR: the variability was higher for positive than for negative perturbations, but again only outside the range ±50 ms. The asymmetric behavior of the PCR led the author to the interpretation that it is easier to delay than to advance a tap in response to a perturbation (Repp, 2011a).

So far, asymmetries have been largely reported for phase shifts only, and particularly for large perturbation sizes or asynchronies. It is remarkable that these findings have not yet been reflected in substantial changes to the linear models, despite the common observation that the estimates for the linear parameters depend for instance on perturbation magnitude (see below). To our knowledge, nonlinear models within the internal timekeeper framework have only been proposed to account for saturation of the response and only in an isochronous task. Engbert et al. (2002) proposed a nonlinear model to explain a small subset of the data in an isochronous finger tapping task. They showed that an error correction model with a saturating function tanh(en) was consistent with the experimental results, and interpreted it as a saturation in the subject’s response when the synchrony error en is large.

The nonlinear behavior, however, seems pretty robust, and sooner or later the models will have to address it (Repp, 2011a). Up to now, the linear approximation appeared to be valid for small perturbation sizes. However, in this work we demonstrate asymmetric responses to step-change perturbations for perturbation magnitudes smaller than 10% of a 500 ms period, and accordingly propose a nonlinear (quadratic) model. There is an important qualitative difference between a quadratic and a linear model—after symmetric perturbations, a linear model can only yield symmetric responses, showing an overshoot either to both perturbations or to none (Loehr et al., 2011). This difference cannot be addressed quantitatively with a better fitting of a linear model, and calls for a revision of the validity of linear models within the usual range of small perturbation magnitudes.

Perturbation experiments are usually performed to probe the response of the system, most notably in the form of either a “step change”, or a “phase shift”, or an “event-onset shift” (see Fig. 2), where both the magnitude of the perturbation and the time of occurrence are unexpected. A surprisingly common approach in the field—regardless of the chosen framework, whether timekeeper- or oscillator-based—is to fit the model’s parameter values separately to different conditions, thus yielding different parameter estimates for different perturbation types and even for different perturbation magnitudes within the same perturbation type (see, e.g., Thaut et al., 1998, Repp, 2001b, Schulze et al., 2005, Large et al., 2002—for a recent exception to this common choice, see Loehr et al., 2011). Although the parameter names are the same within each model, after separate fitting the coefficient of the phase or period correction may be small for some perturbations and large for other perturbations, effectively changing the model’s correction strategy and thus the interpretation of the data. For instance, a study that explicitly suggested separate strategies for correcting large and small step-change perturbations is that of Thaut et al. (1998). The authors found a significant difference in their model’s fitted parameter values for large and small step-change perturbations, including a huge difference of two orders of magnitude between the extreme values of the fitted β (period correction), ranging from −0.496 to −0.006, effectively making the proposed period correction term disappear for some perturbation magnitudes. This, together with the observation of apparently qualitatively and quantitatively different time evolutions for the experimental series, led the authors to state, “the observed multiple synchronization strategies are expressed in our brief mathematical model through adjustments in the equation constants”.

Within the dual-process model (Mates, 1994a, Mates, 1994b), the proposed period correction process was shown to be dependent on the subject’s awareness of a tempo perturbation. Repp, 2001b, Repp and Keller, 2004 fitted the model separately to each perturbation size and found a significant difference between the values of the period correction coefficient β contingent on the detection responses. Awareness of the perturbation is undoubtely an important factor of the underlying mechanism responsible for the synchronization behavior. However, the subject is still able to achieve synchronization whether being aware or not. Thus, the fact that a linear compound model (period + phase correction) can reproduce the observed behavior only after separate fitting to different perturbation size conditions leaves the door open for attempting to construct a single nonlinear model that can uniformly account for the data of all conditions, as discussed in the following.

A related usual practice within the dual-process error correction framework is to consider the complete model when step-change perturbations are studied (both phase and period correction, e.g., Repp (2001b)), but only one equation for phase-shift and event-onset shift perturbations (e.g., Repp, Keller, & Jacoby, 2012). In the second condition, this is equivalent to setting all parameters of the period correction equation to zero in advance. An important shortcoming of this practice of separate fitting is that it does not offer any explanation as to how the subject would choose to “activate” the correct equation (i.e., mechanism) at the beginning of the trial, since he/she is unaware of the magnitude of the upcoming perturbation and thus it would be impossible to “shift gears” in advance. Even in the hypothetical case of the subject being able to immediately trigger the correct timing mechanism right after the first perturbed step (which is unlikely since all three types of perturbation start with the same first perturbed step, see Fig. 2), a second mechanism would be needed in addition to the usual error correction to make the choice. Therefore, the procedure of separate fitting implicitly and necessarily assumes that there should be an additional control mechanism for quickly selecting the appropriate set of parameter values, i.e., selecting different correction mechanisms or strategies described by different parameter values. Indeed, Schulze et al. (2005) estimated their model parameters separately both per tempo and tempo change condition and noted, “there must exist additional control mechanisms that determine when the period adjustment mechanism is started and stopped (e.g., by setting the period correction gain)”.

A more parsimonious account of the behavior would avoid considering either different equations for different perturbations, or separate fitting, or any additional control mechanism, and instead describe the response to all perturbation types and magnitudes as the dynamical response of a fixed, possibly compound, model. As Thaut et al. (1998) posed it regarding step changes, “a self-regulatory model should be designed in which a given set of parameters in a difference equation simulates synchronization responses to step changes of all sizes […] the ensuing model would have to be one describing a nonlinear system.” Thus, some fundamental questions arise: Can we describe the behavior with a simple model and a single set of parameter values, and no additional control mechanisms? Can all these various, seemingly different observed responses be part of a broader spectrum of possible responses of a single system? Even if a two-equation model is considered (as in Mates’ model), can all the various responses be correctly described without turning any of the equations off depending on the perturbation type or size?

The sensorimotor synchronization behavior is likely to draw on several distinct neural processes, namely time perception, interval comparison, error detection, time production, and motor execution (Repp, 2005). The questions posed above relate to whether this likely superposition of neural processes leads to different mechanisms for different perturbations, requiring an additional control mechanism for making the choice, or whether the whole behavior can be interpreted as the result of a compound mechanism represented by a relatively simple model with a fixed number of terms and a single set of parameter values.

In this work we search for both theoretical and experimental evidence supporting the hypothesis of a single underlying compound mechanism for the sensorimotor synchronization behavior in humans. By this we mean that, although several distinct neural processes are probably involved as we pointed out above, the error-correction mechanism resulting from the interplay of such processes can be interpreted as a single entity, as opposed for instance to separate mechanisms for correcting perturbations of different types or magnitudes. Based on the dynamical constraints that the observed behavior imposes on the possible models, we propose a mathematical model for the error correction function f (Eq. (1)), without assuming the existence of an internal clock of any particular kind, or any other hypothesis about the actual neural mechanism in charge of achieving average synchronization. We search for a unified formal framework in which the model accounts for average data from the three most common types of perturbation experiments and for all studied perturbation magnitudes with a single set of parameter values and a fixed number of equations and terms. Although not a proof, the success of such a unification effort would be suggestive evidence for the nonlinearity and the existence of an underlying compound mechanism.

Section snippets

General considerations

We choose the observable synchronization error en as our main variable because of its fundamental nature (Chen et al., 1997), and propose an error correction model in the form of a map like Eq. (1)—that is, we propose the shape of the function f. We assume as our working hypotheses that it is possible to identify a deterministic component within the general mechanism of error correction (i.e., we assume the usual separation between a deterministic rule and noise), and that the qualitative

Participants

The experiment consisted of a finger-tapping task with step-change perturbations. The participants were 10 volunteers (one female, ages 18–36), with no previous training in finger-tapping tasks. Most participants had substantial musical training (7 years or more; only two had less than 3 years). Five of them played percussion.

Materials and equipment

The recording of finger tapping with a standard keyboard and computer has a number of drawbacks, such as delays due to multitasking operating systems, delays due to the

Step-change perturbations

Our experimental time series are displayed in Fig. 6, along with simulations of our model using the parameter values fitted in Fig. 5B (see Table 1 and Section 3).

The qualitative agreement between data and model is remarkable, particularly because all simulations were performed with a single set of parameter values and because the model was fitted to the ±50 ms perturbations only. The response to positive perturbations (top row) displays an overshoot which is larger for larger perturbation

Discussion and conclusions

Although pioneering work on timing goes back as far as the 19th century, e.g., Stevens (1886), the representation of time in our brain has only recently begun to rise as a fundamental issue in neuroscience (Ivry & Schlerf, 2008). During the last decade, several studies have been published on the theoretical and experimental aspects of neural timing (Buhusi and Meck, 2005, Ivry and Schlerf, 2008, Mauk and Buonomano, 2004) and sensorimotor synchronization (for a thorough review see Repp, 2005).

Acknowledgments

We thank Dean V. Buonomano, Mariano Sigman, Juan Kamienkowski, Manuel C. Eguía, Andrés Benavides, and Guillermo Solovey for helpful discussions and comments. We thank Bruno Repp, Nori Jacoby, and an anonymous reviewer for detailed suggestions to improve the manuscript. This work was supported by CONICET and ANPCyT PICT-2007-881 (Argentina), and the Fulbright Commission (USA-Argentina).

References (58)

  • P. Praamstra et al.

    Neurophysiological correlates of error correction in sensorimotor-synchronization

    NeuroImage

    (2003)
  • J. Pressing

    Error correction processes in temporal pattern production

    Journal of Mathematical Psychology

    (1998)
  • B.H. Repp

    Processes underlying adaptation to tempo changes in sensorimotor synchronization

    Human Movement Science

    (2001)
  • B.H. Repp

    Phase correction in sensorimotor synchronization: Nonlinearities in voluntary and involuntary responses to perturbations

    Human Movement Science

    (2002)
  • B.H. Repp et al.

    Quantifying phase correction in sensorimotor synchronization: Empirical comparison of three paradigms

    Acta Psychologica

    (2012)
  • A. Schachner et al.

    Spontaneous motor entrainment to music in multiple vocal mimicking species

    Current Biology

    (2009)
  • G. Schöner

    Timing, clocks, and dynamical systems

    Brain and Cognition

    (2002)
  • D. Vorberg et al.

    Linear phase-correction in synchronization: predictions, parameter estimation, and simulations

    Journal of Mathematical Pschology

    (2002)
  • C.V. Buhusi et al.

    What makes us tick? Functional and neural mechanisms of interval timing

    Nature Reviews Neuroscience

    (2005)
  • Y. Chen et al.

    Long memory processes (1/fa type) in human coordination

    Physical Review Letters

    (1997)
  • M.F. Del Olmo et al.

    Role of the cerebellum in externally paced rhythmic finger movements

    Journal of Neurophysiology

    (2008)
  • C. Drake et al.

    Tempo sensitivity in auditory sequences: Evidence for a multiple-look model

    Perception and Psychophysics

    (1993)
  • K. Drewing et al.

    Reduced timing variability during bimanual coupling: A role for sensory information

    The Quarterly Journal of Experimental Psychology

    (2003)
  • C.B. Gibbs

    Probability learning in step-input tracking

    British Journal of Psychology

    (1965)
  • J. Gross et al.

    The neural basis of intermittent motor control in humans

    Proceedings of the National Academy of Science Uniteed States of America

    (2002)
  • H. Haken et al.

    A theoretical model of phase transitions in human hand movements

    Biological Cybernetics

    (1985)
  • D. Hary et al.

    Synchronizing human movement with an external clock source

    Biological Cybernetics

    (1987)
  • D. Hary et al.

    Temporal tracking and synchronization strategies

    Human Neurobiology

    (1987)
  • I.J. Hirsh

    Auditory perception of temporal order

    Journal of the Acoustical Society of America

    (1959)
  • Cited by (23)

    View all citing articles on Scopus
    1

    Current address: Departamento de Física, Universidad de Buenos Aires, Pabellón I Ciudad Universitaria, Buenos Aires C1428EHA, Argentina.

    View full text