Elsevier

NeuroImage

Volume 39, Issue 3, 1 February 2008, Pages 1383-1395
NeuroImage

Neural substrates of visuomotor learning based on improved feedback control and prediction

https://doi.org/10.1016/j.neuroimage.2007.09.062Get rights and content

Abstract

Motor skills emerge from learning feedforward commands as well as improvements in feedback control. These two components of learning were investigated in a compensatory visuomotor tracking task on a trial-by-trial basis. Between-trial learning was characterized with a state–space model to provide smoothed estimates of feedforward and feedback learning, separable from random fluctuations in motor performance and error. The resultant parameters were correlated with brain activity using magnetic resonance imaging. Learning related to the generation of a feedforward command correlated with activity in dorsal premotor cortex, inferior parietal lobule, supplementary motor area and cingulate motor area, supporting a role of these areas in retrieving and executing a predictive motor command. Modulation of feedback control was associated with activity in bilateral posterior superior parietal lobule as well as right ventral premotor cortex. Performance error correlated with activity in a widespread cortical and subcortical network including bilateral parietal, premotor and rostral anterior cingulate cortex as well as the cerebellar cortex. Finally, trial-by-trial changes of kinematics, as measured by mean absolute hand acceleration, correlated with activity in motor cortex and anterior cerebellum. The results demonstrate that incremental, learning-dependent changes can be modeled on a trial-by-trial basis and neural substrates for feedforward control of novel motor programs are localized to secondary motor areas.

Introduction

Learning a new motor skill is based on hidden, intrinsic changes within the nervous system that are inferred by measurable changes of performance (Schmidt, 1975). Functional imaging provides an opportunity to identify these intrinsic changes within the brain. Previous imaging studies establish that activity in the brain changes with learning, and that this change correlates with numerous measures of competency including time on task (Grafton et al., 1992, Grafton et al., 1994, Grafton et al., 2001, van Mier et al., 1998), tracking error (Miall et al., 2001, Diedrichsen et al., 2005, Miall and Jenkinson, 2005), speed and reaction time. These findings establish the feasibility of using imaging with blood oxygen level-dependent (BOLD) magnetic resonance imaging (MRI) to capture learning-related changes in the brain, but also highlight the need to distinguish different cognitive, kinematic or control mechanisms that lead to these BOLD signal changes.

Here we used a compensatory tracking task in which participants had to counteract the influence of a repeating input function. As learning progressed, participants became more adept at keeping a cursor in a goal region. For this task, we can distinguish at least four separable but not necessarily independent components that could lead to learning-related BOLD signal changes. (1) Motor learning is by definition accompanied by a change in motor output. Given the known correlations between movement kinematics and brain activity (Dettmers et al., 1996, Turner et al., 1998, Turner et al., 2003), one is faced with the problem of distinguishing changes of brain activity due to learning from changing movement kinematics. It is possible to manipulate tasks such that kinematics remain constant, through use of dual tasks (Seidler et al., 2002). However, these data can be difficult to generalize to other forms of learning as there is evidence that neural substrates of skill learning can differ enormously in the context of training under single and dual tasks (Grafton et al., 1995, Grafton et al., 1998, Hazeltine et al., 1997). A second approach is to identify changes of network dynamics, rather than local BOLD magnitude, to characterize learning (Tunik et al., 2007). A third, model-based approach is to parameterize learning and show that these parameters are uncorrelated with low-level kinematic features. Then the model parameters can be correlated with brain activity. This is the general strategy of the current study. (2) Performance error is used to correct the on-going movement as well as to serve as signal to change the movement plan of the next movement (Diedrichsen et al., 2005, Suminski et al., 2007). There is a need to distinguish brain activity associated with the perception and correction of error within each trial from modifications across future trials (Fisher et al., 2000). In the current study, within-trial error was correlated directly with brain activity on a trial-by-trial basis and between-trial effects were incorporated in the learning model. (3) Performance can improve by changing how feedback information is used, i.e., through a change of the feedback control policy (Garvey, 1960, Fuchs, 1962, Fitts, 1964). This improvement can occur irrespective of the ability generate a predictive motor command. For example, performance can improve when tracking randomly moving targets by adopting an optimal feedback control policy that takes into account the inherent temporal delays (Foulkes and Miall, 2000). (4) Predictive or feedforward control can occur by perceptual learning of the trajectory of a target or by motor learning of a particular required movement (Adams, 1987). Both perceptual and motor learning result in predictive control but they have fundamentally different patterns of generalization and neural substrates (Grafton et al., 2001). These different contributions to learning motivated the current experimental approach. The subjects used a frictionless knob and made pronation–supination movements of the forearm to keep a cursor centered while an unknown but consistently repeating input function moved the cursor off the midline. Subjects learned a novel and unique spatio-temporal motor pattern necessary to counteract the hidden input that perturbed the position of a cursor.

We distinguished contributions based on feedforward learning from feedback corrections driven by visual feedback using the temporal delay of the response. Feedback corrections were modeled as a response to position and velocity of the feedback cursor with a 150-ms delay. In contrast, predictive responses were modeled as being related to the input function without time delay. To determine whether these feedforward responses were learned as a motor program or through perceptual learning of the input pattern, we used a transfer task after 4 days of training in which the learned skill had to be generalized to a new control situation with either an identical motor response or perceptual pattern instructing the desired movement. As shown below, generalization only occurred with transfer to control situations requiring identical movement.

Control parameters were estimated to describe the strength of feedforward and feedback control on every single trial. To distinguish these learning-related changes from trial-by-trial variations of the size of the visual error and the actual motor output, a state–space model that provided optimally smoothed estimates of the control parameter was used. In control engineering, a state–space representation is a mathematical model of a physical system described as a set of input, output and state variables related by first-order differential equations. State–space models can be used to represent the output of a visuomotor control system as a function of the input and of the hidden state of the system at a particular moment in time, i.e., within a given motor trial. They can also be used to describe the temporal dynamics of how the hidden state develops over time, i.e., how the visuomotor system learns from trial to trial. Similar approaches have been used to model learning and generalization in a force-field reaching task (Thoroughman and Shadmehr, 2000, Donchin et al., 2003, Smith et al., 2004) as well as during sensorimotor adaptation to altered visual feedback (Cheng and Sabes, 2006). A general tenet is that the parameter changes related to learning are relatively smooth (Baddeley et al., 2003), while changes in motor error and motor output fluctuate, uncorrelated, across trials.

Previous state–space models of motor learning have characterized the success of a subject in learning to adjust the execution of a familiar movement, such as pointing to a target in the setting of altered dynamics or visual feedback. In contrast, the current experiment required subjects to learn an unfamiliar movement pattern in the setting of stable dynamics and normal visual feedback. Feedforward learning in the present case represents the capacity to retrieve a newly acquired motor program. The parameter estimate of feedback learning in the present study is an abstract metric describing the ability of a subject to modify their use of feedback information to improve performance. There are many ways a person might accomplish this, including better use of visual attention, smoother movements or enhanced understanding of the controller dynamics. The current measure does not distinguish these sources of improved use of feedback information.

The experimental design also allowed for an assessment of standard measures of performance for each trial. Motor error was measured as the mean absolute position error over a trial and motor output was measured as the mean absolute hand acceleration. Other measures of the amount of motor output could also potentially be used, such as average velocity or force, but these measures typically correlate among each other and with hand acceleration. Note that error and hand acceleration are sensitive to trial-by-trial noise whereas our state–space model estimates of feedback and feedforward learning were calculated to be independent of this non-specific performance variability. We then correlated these four measures with MRI BOLD measures of brain activity on a trial-by-trial basis. In this way, brain activity related to both learning and performance could be localized. The core hypotheses in the present paper tested if there were different brain correlates of feedback policy and feedforward learning and if these would be localized in the same brain areas driven by trial to trial differences of kinematics or error.

While the relationship between brain activity, kinematics and error are well established, the brain systems related to feedforward and feedback control have not been defined. One strategy for distinguishing brain systems underlying feedback and feedforward control used Fitt’s law in a pointing task. With this construct and fMRI, it was proposed that there is greater feedback control when targets are small and greater feedforward control with “ballistic” movements towards large targets. fMRI identified areas more strongly activated with large targets (and presumably feedforward control) included primary motor cortex, premotor cortex and the basal ganglia. Areas associated with more difficult targets (presumably with greater feedback control) included ipsilateral sensorimotor cortex, multiple cerebellar regions and the thalamus (Seidler et al., 2004). While feedback control is likely to increase for small targets, the assumption that feedforward control would change at all is untested. Subjects could use the same feedforward program, but at a higher velocity. Thus, differences in motor output, movement speed, etc., complicate the interpretation of these results and underscore the need for alternative methods to distinguish feedforward and feedback control.

Changes in feedforward control have also been studied using visuomotor adaptation. Learning to reach with a rotated joystick induces widespread changes of activity including increased activity in the cerebellum (Imamizu et al., 2000, Della-Maggiore and McIntosh, 2005, Graydon et al., 2005) and basal ganglia (Seidler et al., 2006). However, many of these studies have not controlled for other factors, such as movement error, eye movements and hand motor output. When controlling for these factors, the acquisition of a new visuomotor mapping appears to be connected with increased activity in the right parietal cortex (Krakauer et al., 2004). Finally, it has been proposed that premotor and inferior parietal cortex or basal ganglia retrieve and manipulate specific motor programs, i.e., “action vocabularies” used in feedforward control (Graybiel, 2000, Rizzolatti and Craighero, 2004, Johnson-Frey et al., 2005).

Section snippets

Task and performance measurement

An optically encoding MRI compatible knob with a screwdriver handle and minimal resistive torque sampled rotational position of the forearm at 200 Hz. Subjects viewed a computer monitor projected by LCD onto a rear projection screen at the head of the bore via a mirror mounted to the head coil. The screen was colored dark gray and bisected by a stationary vertical red bar that served as the target. The knob controlled the lateral position of a second, thick yellow vertical bar. The knob

Behavioral results

Positional error for both leftward and rightward movements decreased significantly over the 4 days of training, typical of visuomotor skill acquisition (repeated measures ANOVA, with factors for direction: not significant (NS), trial: F = 13.03, df = 207, p < 0.0001, and direction × trial interaction (NS); Fig. 3A). Mean absolute hand acceleration also decreased significantly over time (Fig. 3B; repeated measures ANOVA with factors for direction: NS, trial: F = 22.204, df = 8,207, p < 0.0001, and their

Discussion

We examined the formation of a pair of simple symmetric motor programs of fixed duration and amplitude. Detailed analysis by our learning model showed that improvement occurred because of both adjustments in feedback control as well as the acquisition of feedforward control. The parameters describing feedback and feedforward learning showed only weak trial-by-trial correlations for 9 of 10 participants, allowing us to identify underlying neural substrates for these two sources of learning.

The

Acknowledgments

The authors thank Antonia Hamilton and Eugene Tunik for their helpful comments. Current affiliations: Scott T. Grafton MD, Department of Psychology and The Sage Center for the Study of the Mind, University of California, Santa Barbara, CA. Jack Van Horn, Laboratory of Neuro Imaging, UCLA School of Medicine, Los Angeles CA. Jörn Diedrichsen, School of Psychology, University of Wales, Bangor, Wales. Grants: Supported by PHS grant NS33504.

References (86)

  • M.F. Rushworth et al.

    The left hemisphere and the selection of learned actions

    Neuropsychologia

    (1998)
  • M.F. Rushworth et al.

    Action sets and decisions in the medial frontal cortex

    Trends Cogn. Sci.

    (2004)
  • J.D. Schmahmann et al.

    Three-dimensional MRI atlas of the human cerebellum in proportional stereotaxic space

    NeuroImage

    (1999)
  • R.D. Seidler et al.

    Feedforward and feedback processes in motor control

    NeuroImage

    (2004)
  • S.P. Wise et al.

    Arbitrary associations between antecedents and actions

    Trends Neurosci.

    (2000)
  • M.W. Woolrich et al.

    Multilevel linear modelling for FMRI group analysis using Bayesian inference

    NeuroImage

    (2004)
  • M.W. Woolrich et al.

    Temporal autocorrelation in univariate linear modeling of FMRI data

    NeuroImage

    (2001)
  • J.A. Adams

    Historical review and appraisal of research on the learning, retention and transfer of human motor skills

    Psychol. Bull.

    (1987)
  • D.A. Backus et al.

    Neural activity correlated with the preparation and execution of visually guided arm movements in the cingulate motor area of the monkey

    Exp. Brain Res.

    (2001)
  • R.J. Baddeley et al.

    System identification applied to a visuomotor task: near-optimal human performance in a noisy changing task

    J. Neurosci.

    (2003)
  • W.L. Bryan et al.

    Studies in the physiology and psychology of telegraphic language

    Psychol. Rev.

    (1897)
  • W.L. Bryan et al.

    Studies on the telegraphic language, the acquisition of a hierarchy of habits

    Psychol. Rev.

    (1899)
  • C.S. Carter et al.

    Anterior cingulate cortex, error detection, and the online monitoring of performance

    Science

    (1998)
  • S. Cheng et al.

    Modeling sensorimotor learning with linear dynamical systems

    Neural Comput.

    (2006)
  • D.M. Clower et al.

    Basal ganglia and cerebellar inputs to ‘AIP’

    Cereb. Cortex

    (2005)
  • M. Corbetta et al.

    Control of goal-directed and stimulus-driven attention in the brain

    Nat. Rev., Neurosci.

    (2002)
  • D.J. Crammond et al.

    Prior information in motor and premotor cortex: activity during the delay period and effect on pre-movement activity

    J. Neurophysiol.

    (2000)
  • V. Della-Maggiore et al.

    Time course of changes in brain activity and functional connectivity associated with long-term adaptation to a rotational transformation

    J. Neurophysiol.

    (2005)
  • J. Diedrichsen et al.

    Neural correlates of reach errors

    J. Neurosci.

    (2005)
  • O. Donchin et al.

    Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control

    J. Neurosci.

    (2003)
  • H. Duvernoy

    The Human Brain. Surface, Three-Dimensional Sectional Anatomy and MRI

    (1991)
  • B.E. Fisher et al.

    Deficits in compensatory trajectory adjustments after unilateral sensorimotor stroke

    Exp. Brain Res.

    (2000)
  • A. Floyer-Lea et al.

    Distinguishable brain activation networks for short- and long-term motor skill learning

    J. Neurophysiol.

    (2005)
  • L. Fogassi et al.

    Parietal lobe: from action organization to intention understanding

    Science

    (2005)
  • A.J. Foulkes et al.

    Adaptation to visual feedback delays in a human manual tracking task

    Exp. Brain Res.

    (2000)
  • A.H. Fuchs

    The progression–regression hypothesis in perceptual–motor skill learning

    J. Exp. Psychol.

    (1962)
  • W.D. Garvey

    A comparison of the effects of training and secondary tasks on tracking behavior

    J. Appl. Psychol.

    (1960)
  • S.T. Grafton et al.

    Functional anatomy of sequence learning in normal humans

    J. Cogn. Neurosci.

    (1995)
  • S.T. Grafton et al.

    Abstract and effector-specific representations of motor sequences identified with PET

    J. Neurosci.

    (1998)
  • S.T. Grafton et al.

    Functional anatomy of human procedural learning determined with regional cerebral blood flow and PET

    J. Neurosci.

    (1992)
  • S.T. Grafton et al.

    Motor learning of compatible and incompatible visuomotor maps

    J. Cogn. Neurosci.

    (2001)
  • S.T. Grafton et al.

    Functional imaging of procedural motor learning: relating cerebral blood flow with individual subject performance

    Hum. Brain Mapp.

    (1994)
  • K.Y. Haaland et al.

    Neural representations of skilled movement

    Brain

    (2000)
  • Cited by (0)

    View full text