Two distinct modes of control for object-directed action

https://doi.org/10.1016/S0079-6123(03)14409-3Get rights and content

Abstract

There are multiple routes from vision to action that play a role in the production of visually guided reaching and grasping. What remain to be resolved, however, are the conditions under which these various routes are recruited in the generation of actions and the nature of the information they convey. We argue in this chapter that the production of real-time actions to visible targets depends on pathways that are separate from those mediating memory-driven actions. Furthermore, the transition from real-time to memory-driven control occurs as soon as the intended target is no longer visible. Real-time movements depend on pathways from the early visual areas through to relatively encapsulated visuomotor mechanisms in the dorsal stream. These dedicated visuomotor mechanisms, together with motor centers in the premotor cortex and brainstem, compute the absolute metrics of the target object and its position in the egocentric coordinates of the effector used to perform the action. Such real-time programming is essential for the production of accurate and efficient movements in a world where the location and disposition of a goal object with respect to the observer can change quickly and often unpredictably. In contrast, we argue that memory-driven actions make use of a perceptual representation of the target object generated by the ventral stream. Unlike the real-time visuomotor mechanisms, perception-based movement planning makes use of relational metrics and scene-based coordinates. Such computations make it possible, however, to plan and execute actions upon objects long after they have vanished from view.

Introduction

When we reach out to grasp an object, the shape of our hand begins to reflect the size, shape, and orientation of the target almost as soon as the movement is initiated (Jeannerod, 1986). The programming of grasping must therefore depend on visual information about the goal object that is garnered before the movement begins. Once the movement is initiated, feedback information is used to adjust the final position and posture of the hand as it closes in on the object (i.e., movement control). There is a good deal of evidence to suggest that the visual processing used in the programming and the control of grasping is quite distinct from the visual processing that supports our perception and recognition of objects (Goodale and Milner, 1992). Much of the evidence for this distinction between what has been called ‘vision-for-action’ and ‘vision-for-perception’ comes from work with neurological patients.

Perhaps the most compelling example is the patient D.F., who developed a profound visual form agnosia as a consequence of an anoxic episode. Even though D.F. shows no perceptual awareness of the form and dimensions of objects, she is able to scale her hand to the size, shape, and orientation of a goal object as she reaches out to pick it up (Goodale et al., 1991, Milner et al., 1991). On the basis of these and related observations, Goodale and Milner (1992) argued that the visual control of grasping and other visually guided movements is mediated by dedicated visuomotor systems in the dorsal stream — a set of visual projections that arise in primary visual cortex and project to areas in the posterior parietal cortex. The perception of objects, they argued, depends on a quite separate set of ventral-stream projections that also arise in primary visual cortex but project instead to the temporal lobe. As it turns out, D.F.'s brain damage is concentrated at the junction of the occipital and temporal lobes, in a region of the human ventral tream that has been shown to be involved in the visual recognition of objects (James et al., in press). It is presumably this selective damage to her ventral stream that has disrupted her ability to perceive the form of objects without interfering with her ability to reach out and grasp objects. In fact, recent neuroimaging studies have revealed relatively normal activation in D.F.'s dorsal stream when she grasps objects that vary in size and orientation (Culham, 2004).

Consistent with these observations, neurological patients with damage to the dorsal stream often demonstrate impaired grasping movements, despite having relatively intact perception of object features (Perenin and Vighetto, 1988, Milner et al., 2001). As soon as their fingers make contact with the target, of course, they can use haptic information to adjust their hand to the correct posture. Many of these patients are also unable to reach in the correct direction to objects placed in different positions in the visual field contralateral to their lesion. But despite exhibiting a clear deficit in the visual control of reaching and grasping (known clinically as ‘optic ataxia’), these same patients have little difficulty describing the orientation, size, shape, and even the relative spatial location of the very objects they are unable to grasp correctly (for review, see Milner and Goodale, 1995).

Evidence for the distinction between ‘vision-for-action’ and ‘vision-for-perception’ also comes from experiments with normal observers. Many studies have reported that the scaling of grip aperture in manual prehension — which is strongly influenced by the size of the target object — is unaffected by size-contrast illusions that, by definition, influence perceptual judgments about the size of the target object (see Carey, 2001 for review). In a recent study, for example, Hu and Goodale (2000) asked participants to estimate the width of a rectangular block that was presented alongside a smaller or larger companion block. As expected, these perceptual estimates were affected by the relative size of the companion block. When accompanied by a larger block, the target block was perceived to be smaller than when it was accompanied by a smaller block. The presence of the companion blocks had no effect, however, on the scaling of the grip aperture when the participants were asked to reach out to pick up the target block. These and other related findings are quite consistent with the idea that separate visual mechanisms mediate object perception and object-directed action.

According to the two-visual-systems account, size-contrast illusions affect perception and not action because the ventral and dorsal streams compute the location, size, and disposition of objects in quite different ways. The perceptual mechanisms in the ventral stream use scene-based frames of reference and relational metrics, computations that provide a rich and detailed representation of the world upon which cognitive processes can operate. Thus, the ventral stream — and therefore perception — falls victim to size-contrast illusions because such illusions engage the obligatory relative-size computations that characterize the operation of this system. In contrast, the visuomotor mechanisms in the dorsal stream use egocentric frames of reference and compute the absolute metrics of the target object, computations that are required for accurate and efficient actions. Thus, actions like grasping are refractory to size-contrast illusions because they are scaled to the real not the relative size of the target object (Goodale and Haffenden, 1998).

It should be noted that a number of studies have reported that grip scaling is affected by size-contrast illusions, in some cases to the same degree as perceptual size judgments (Pavani et al., 1999, Franz et al., 2000, Vishton et al., 1999, Glover and Dixon, 2001). Clearly then, there must be some conditions under which illusions can affect grip scaling. But one has to be careful in interpreting these experiments, since the investigators have not always used tasks that reliably invoke the dedicated visuomotor mechanisms in the dorsal stream. Some, for example, have used two-dimensional stimuli, which are often not treated as real goal objects, causing subjects to generate movements that they would not normally use in picking up objects (Vishton et al., 1999). Others have used illusions that operate extremely early in the visual system and thus affect both dorsal and ventral streams of visual processing (Glover and Dixon, 2001). [For a discussion of these issues, see Haffenden et al. (2001) and Dyde and Milner (2002).]

But in any case the failure to demonstrate a difference in the effects of illusions on visually guided action and visual perception does not seriously challenge the two-visual-systems theory. These negative findings cannot explain, for example, the behavioral dissociation between vision-for-perception and vision-for-action that has been observed in neurological patients. Nor for that matter can they explain why so many other studies have found evidence for this dissociation in normal observers. The findings do suggest, however, that the control of action is not always encapsulated within the visuomotor modules of the dorsal stream and that, under some circumstances, the control of action will make use of perceptual information that is presumably processed in the ventral stream. In this chapter, we explore some of the conditions that determine whether an action will be controlled entirely by the dorsal stream or will instead be influenced by information provided by perceptual mechanisms in the ventral stream. We will suggest that dedicated, real-time visuomotor mechanisms are engaged for the feedforward control of action only at the moment an individual decides to make a goal-directed action, and only if the target is visible.

We shall make an important distinction between two feedforward modes of control for action, which we refer to as planning and programming. In this chapter we will argue that action planning begins as soon as the observer perceives the goal object, and that this action planning depends — at least in part — on the perceptual mechanisms in the ventral stream (see Glover and Dixon, 2001 for a similar argument). We review evidence suggesting that the period of time immediately prior to movement onset is critical for action programming. During this period of time direct retinal information from the goal object is transformed immediately into a metrically accurate movement program. In other words, action programming — quite unlike action planning — occurs in real time. Psychophysical and neuropsychological evidence suggests that real-time action programming depends on the visuomotor mechanisms in the dorsal pathway, rather than the perceptual mechanisms in the ventral pathway that are so critical for action planning. In light of these arguments, one must interpret with caution the results of experiments that use delayed rather than real-time action tasks to explore the cognitive and neural mechanisms of sensorimotor transformation.

Section snippets

The effects of delay on visually guided actions in neurological patients

Studies with neurological patients have demonstrated clear limits on the ability of the isolated dorsal system to guide manual prehension (Fig. 1). For example, D.F. demonstrates extremely poor size scaling of her grip when she reaches to grasp a target object after it has been removed from view, even though she shows good scaling when the object is visible up until the time of movement initiation (Goodale et al., 1994). In fact, when a 2-s delay was introduced between viewing the goal object

Differences in the timing of action and perception

As was mentioned earlier, vision-for-action and vision-for-perception use different metrics and frames of reference. To be able to grasp an object successfully, for example, it is essential that the brain compute the actual (absolute) size of the object and its orientation and position with respect to the observer. Moreover, the information about the orientation and position of the object must be computed in egocentric frames of reference — in other words, in frames of reference that take into

The effects of delay on actions in normal observers

Goodale et al. (1994) showed that the imposition of a delay of 2 s between object viewing and movement execution produced a dramatic alteration in the kinematics of the grasp in normal observers. Not only were movements to the remembered object slower than movements made to a visible object, but the trajectory of the hand was more curvilinear, rising higher above the surface on which the object had been presented. Moreover, the hand did not show the typical overshoot in grip aperture that

Does the dorsal stream have a memory?

As we have just seen, in memory-guided grasping, the motor system must generate a movement program using a stored representation of the target object — a representation that was originally created by perceptual mechanisms in the ventral stream. In real-time grasping, in which fast and metrically accurate movements are generated, the underlying visuomotor computations require direct visual input. It has been argued that the on-line computations that inform the required movement programming have

Comparing visually-guided and memory-guided grasping

We used a size-contrast illusion to assess the contribution of perceptual mechanisms to the control of visually-guided and memory-guided grasping movements. Previous experiments that have examined this question have employed separate blocks of trials for visually-guided and memory-guided responses (Goodale et al., 1994, Hu and Goodale, 2000, Westwood et al., 2000a, Westwood et al., 2000b, Milner et al., 2001). As a consequence, the participants in these experiments could have attended to quite

Off-line versus real-time control of actions

These findings are consistent with those of earlier studies suggesting that perceptual mechanisms in the ventral pathway are invoked for the control of actions to remembered targets — targets that are removed from view before the action is initiated. But, as was discussed earlier, it is not clear from previous studies that perceptual mechanisms are in fact necessary for the control of memory-guided actions. Because memory trials were blocked in earlier studies, it could have been the case that

Acknowledgements

Supported by the Canadian Institutes of Health Research, the Canada Research Chairs Program, and the Wellcome Trust.

References (39)

  • M. Jeannerod

    Mechanisms of visuomotor coordination: a study in normal and brain-damaged subjects

    Neuropsychologia

    (1986)
  • M. Jeannerod et al.

    Impairment of grasping movements following a bilateral posterior parietal lesion

    Neuropsychologia

    (1994)
  • A.D. Milner et al.

    Grasping the past. Delay can improve visuomotor performance

    Curr. Biol.

    (2001)
  • R.E. Passingham et al.

    Contrasting the dorsal and ventral visual systems: guidance of movement versus decision making

    NeuroImage

    (2001)
  • W.F. Asaad et al.

    Task-specific neural activity in the primate prefrontal cortex

    J. Neurophysiol.

    (2000)
  • Culham, J.C. (2004) Human brain imaging reveals a parietal area specialized for grasping. In: N. Kanwisher and J....
  • R.T. Dyde et al.

    Two illusions of perceived orientation: one fools all of the people some of the time; the other fools all of the people all of the time

    Exp. Brain Res.

    (2002)
  • V.H. Franz et al.

    Grasping visual illusions: no evidence for a dissociation between perception and action

    Psychol. Sci.

    (2000)
  • S.R. Glover et al.

    Dynamic illusion effect in a reaching task: evidence for separate visual representations in the planning and control of reaching

    J. Exp. Psychol. Hum. Percept. Perform.

    (2001)
  • Cited by (190)

    • Egocentric and allocentric spatial representations in a patient with Bálint-like syndrome: A single-case study

      2021, Cortex
      Citation Excerpt :

      Her difficulty in providing egocentric judgments is in line with previous studies suggesting a dorsal stream involvement in encoding object location in egocentric coordinates (Committeri et al., 2004; Zaehle et al., 2007; Ruotolo et al., 2019). Consistently with the short duration attributed to egocentric representation in dorsal stream (Goodale et al., 2004), the patient's deficit was restricted to egocentric judgments made after 1.5sec delay. However, GP's egocentric judgments paradoxically improved with the 5sec delay and this improvement made GP's performance comparable to that of the controls.

    View all citing articles on Scopus
    View full text