Neural substrates of visuomotor learning based on improved feedback control and prediction
Introduction
Learning a new motor skill is based on hidden, intrinsic changes within the nervous system that are inferred by measurable changes of performance (Schmidt, 1975). Functional imaging provides an opportunity to identify these intrinsic changes within the brain. Previous imaging studies establish that activity in the brain changes with learning, and that this change correlates with numerous measures of competency including time on task (Grafton et al., 1992, Grafton et al., 1994, Grafton et al., 2001, van Mier et al., 1998), tracking error (Miall et al., 2001, Diedrichsen et al., 2005, Miall and Jenkinson, 2005), speed and reaction time. These findings establish the feasibility of using imaging with blood oxygen level-dependent (BOLD) magnetic resonance imaging (MRI) to capture learning-related changes in the brain, but also highlight the need to distinguish different cognitive, kinematic or control mechanisms that lead to these BOLD signal changes.
Here we used a compensatory tracking task in which participants had to counteract the influence of a repeating input function. As learning progressed, participants became more adept at keeping a cursor in a goal region. For this task, we can distinguish at least four separable but not necessarily independent components that could lead to learning-related BOLD signal changes. (1) Motor learning is by definition accompanied by a change in motor output. Given the known correlations between movement kinematics and brain activity (Dettmers et al., 1996, Turner et al., 1998, Turner et al., 2003), one is faced with the problem of distinguishing changes of brain activity due to learning from changing movement kinematics. It is possible to manipulate tasks such that kinematics remain constant, through use of dual tasks (Seidler et al., 2002). However, these data can be difficult to generalize to other forms of learning as there is evidence that neural substrates of skill learning can differ enormously in the context of training under single and dual tasks (Grafton et al., 1995, Grafton et al., 1998, Hazeltine et al., 1997). A second approach is to identify changes of network dynamics, rather than local BOLD magnitude, to characterize learning (Tunik et al., 2007). A third, model-based approach is to parameterize learning and show that these parameters are uncorrelated with low-level kinematic features. Then the model parameters can be correlated with brain activity. This is the general strategy of the current study. (2) Performance error is used to correct the on-going movement as well as to serve as signal to change the movement plan of the next movement (Diedrichsen et al., 2005, Suminski et al., 2007). There is a need to distinguish brain activity associated with the perception and correction of error within each trial from modifications across future trials (Fisher et al., 2000). In the current study, within-trial error was correlated directly with brain activity on a trial-by-trial basis and between-trial effects were incorporated in the learning model. (3) Performance can improve by changing how feedback information is used, i.e., through a change of the feedback control policy (Garvey, 1960, Fuchs, 1962, Fitts, 1964). This improvement can occur irrespective of the ability generate a predictive motor command. For example, performance can improve when tracking randomly moving targets by adopting an optimal feedback control policy that takes into account the inherent temporal delays (Foulkes and Miall, 2000). (4) Predictive or feedforward control can occur by perceptual learning of the trajectory of a target or by motor learning of a particular required movement (Adams, 1987). Both perceptual and motor learning result in predictive control but they have fundamentally different patterns of generalization and neural substrates (Grafton et al., 2001). These different contributions to learning motivated the current experimental approach. The subjects used a frictionless knob and made pronation–supination movements of the forearm to keep a cursor centered while an unknown but consistently repeating input function moved the cursor off the midline. Subjects learned a novel and unique spatio-temporal motor pattern necessary to counteract the hidden input that perturbed the position of a cursor.
We distinguished contributions based on feedforward learning from feedback corrections driven by visual feedback using the temporal delay of the response. Feedback corrections were modeled as a response to position and velocity of the feedback cursor with a 150-ms delay. In contrast, predictive responses were modeled as being related to the input function without time delay. To determine whether these feedforward responses were learned as a motor program or through perceptual learning of the input pattern, we used a transfer task after 4 days of training in which the learned skill had to be generalized to a new control situation with either an identical motor response or perceptual pattern instructing the desired movement. As shown below, generalization only occurred with transfer to control situations requiring identical movement.
Control parameters were estimated to describe the strength of feedforward and feedback control on every single trial. To distinguish these learning-related changes from trial-by-trial variations of the size of the visual error and the actual motor output, a state–space model that provided optimally smoothed estimates of the control parameter was used. In control engineering, a state–space representation is a mathematical model of a physical system described as a set of input, output and state variables related by first-order differential equations. State–space models can be used to represent the output of a visuomotor control system as a function of the input and of the hidden state of the system at a particular moment in time, i.e., within a given motor trial. They can also be used to describe the temporal dynamics of how the hidden state develops over time, i.e., how the visuomotor system learns from trial to trial. Similar approaches have been used to model learning and generalization in a force-field reaching task (Thoroughman and Shadmehr, 2000, Donchin et al., 2003, Smith et al., 2004) as well as during sensorimotor adaptation to altered visual feedback (Cheng and Sabes, 2006). A general tenet is that the parameter changes related to learning are relatively smooth (Baddeley et al., 2003), while changes in motor error and motor output fluctuate, uncorrelated, across trials.
Previous state–space models of motor learning have characterized the success of a subject in learning to adjust the execution of a familiar movement, such as pointing to a target in the setting of altered dynamics or visual feedback. In contrast, the current experiment required subjects to learn an unfamiliar movement pattern in the setting of stable dynamics and normal visual feedback. Feedforward learning in the present case represents the capacity to retrieve a newly acquired motor program. The parameter estimate of feedback learning in the present study is an abstract metric describing the ability of a subject to modify their use of feedback information to improve performance. There are many ways a person might accomplish this, including better use of visual attention, smoother movements or enhanced understanding of the controller dynamics. The current measure does not distinguish these sources of improved use of feedback information.
The experimental design also allowed for an assessment of standard measures of performance for each trial. Motor error was measured as the mean absolute position error over a trial and motor output was measured as the mean absolute hand acceleration. Other measures of the amount of motor output could also potentially be used, such as average velocity or force, but these measures typically correlate among each other and with hand acceleration. Note that error and hand acceleration are sensitive to trial-by-trial noise whereas our state–space model estimates of feedback and feedforward learning were calculated to be independent of this non-specific performance variability. We then correlated these four measures with MRI BOLD measures of brain activity on a trial-by-trial basis. In this way, brain activity related to both learning and performance could be localized. The core hypotheses in the present paper tested if there were different brain correlates of feedback policy and feedforward learning and if these would be localized in the same brain areas driven by trial to trial differences of kinematics or error.
While the relationship between brain activity, kinematics and error are well established, the brain systems related to feedforward and feedback control have not been defined. One strategy for distinguishing brain systems underlying feedback and feedforward control used Fitt’s law in a pointing task. With this construct and fMRI, it was proposed that there is greater feedback control when targets are small and greater feedforward control with “ballistic” movements towards large targets. fMRI identified areas more strongly activated with large targets (and presumably feedforward control) included primary motor cortex, premotor cortex and the basal ganglia. Areas associated with more difficult targets (presumably with greater feedback control) included ipsilateral sensorimotor cortex, multiple cerebellar regions and the thalamus (Seidler et al., 2004). While feedback control is likely to increase for small targets, the assumption that feedforward control would change at all is untested. Subjects could use the same feedforward program, but at a higher velocity. Thus, differences in motor output, movement speed, etc., complicate the interpretation of these results and underscore the need for alternative methods to distinguish feedforward and feedback control.
Changes in feedforward control have also been studied using visuomotor adaptation. Learning to reach with a rotated joystick induces widespread changes of activity including increased activity in the cerebellum (Imamizu et al., 2000, Della-Maggiore and McIntosh, 2005, Graydon et al., 2005) and basal ganglia (Seidler et al., 2006). However, many of these studies have not controlled for other factors, such as movement error, eye movements and hand motor output. When controlling for these factors, the acquisition of a new visuomotor mapping appears to be connected with increased activity in the right parietal cortex (Krakauer et al., 2004). Finally, it has been proposed that premotor and inferior parietal cortex or basal ganglia retrieve and manipulate specific motor programs, i.e., “action vocabularies” used in feedforward control (Graybiel, 2000, Rizzolatti and Craighero, 2004, Johnson-Frey et al., 2005).
Section snippets
Task and performance measurement
An optically encoding MRI compatible knob with a screwdriver handle and minimal resistive torque sampled rotational position of the forearm at 200 Hz. Subjects viewed a computer monitor projected by LCD onto a rear projection screen at the head of the bore via a mirror mounted to the head coil. The screen was colored dark gray and bisected by a stationary vertical red bar that served as the target. The knob controlled the lateral position of a second, thick yellow vertical bar. The knob
Behavioral results
Positional error for both leftward and rightward movements decreased significantly over the 4 days of training, typical of visuomotor skill acquisition (repeated measures ANOVA, with factors for direction: not significant (NS), trial: F = 13.03, df = 207, p < 0.0001, and direction × trial interaction (NS); Fig. 3A). Mean absolute hand acceleration also decreased significantly over time (Fig. 3B; repeated measures ANOVA with factors for direction: NS, trial: F = 22.204, df = 8,207, p < 0.0001, and their
Discussion
We examined the formation of a pair of simple symmetric motor programs of fixed duration and amplitude. Detailed analysis by our learning model showed that improvement occurred because of both adjustments in feedback control as well as the acquisition of feedforward control. The parameters describing feedback and feedforward learning showed only weak trial-by-trial correlations for 9 of 10 participants, allowing us to identify underlying neural substrates for these two sources of learning.
The
Acknowledgments
The authors thank Antonia Hamilton and Eugene Tunik for their helpful comments. Current affiliations: Scott T. Grafton MD, Department of Psychology and The Sage Center for the Study of the Mind, University of California, Santa Barbara, CA. Jack Van Horn, Laboratory of Neuro Imaging, UCLA School of Medicine, Los Angeles CA. Jörn Diedrichsen, School of Psychology, University of Wales, Bangor, Wales. Grants: Supported by PHS grant NS33504.
References (86)
- et al.
General multilevel linear modeling for group analysis in FMRI
NeuroImage
(2003) - et al.
Quantitative comparison of functional magnetic resonance imaging with positron emission tomography using a force-related paradigm
NeuroImage
(1996) - et al.
Role of the striatum, cerebellum and frontal lobes in the automatization of a repeated visuomotor sequence of movements
Neuropsychologia
(1998) - et al.
Motor areas in the frontal lobe of the primate
Physiol. Behav.
(2002) Perceptual–motor skill learning
- et al.
Motor functions of the parietal lobe
Curr. Opin. Neurobiol.
(2005) The basal ganglia
Curr. Biol.
(2000)- et al.
Learning-related fMRI activation associated with a rotational visuo-motor transformation
Brain Res. Cogn. Brain Res.
(2005) The assessment and analysis of handedness: the Edinburgh inventory
Neuropsychologia
(1971)- et al.
Cognitive neuroscience: resolving conflict in and over the medial frontal cortex
Curr. Biol.
(2005)