Human gaze behaviour during action execution and observation
Introduction
Human gaze behaviour has been studied in various natural activities (Hollands et al., 2002, Land and Furneaux, 1997, Land et al., 1999), revealing gaze shifts and fixations to be commonly proactive. A detailed study of gaze-hand coordination during object manipulation (Johansson, Westling, Backstrom, & Flanagan, 2001) revealed that subjects almost exclusively fixated landmarks critical for the control of the task. By contrast, the hand and the moving object were never fixated. Gaze was anticipating hand-object movements to the forthcoming landmarks. Once fixated, gaze remained at a landmark till the hand-object was reaching it (in the case of targets or contact sites) or passing by successfully (in case of obstacles).
In a subsequent study, Flanagan and Johansson (2003) compared eye-hand coordination during the active execution of a block stacking task and during observation of the same task performed by another person. Interestingly, gaze behaviour was proactive as well during action execution as during action observation. The authors speculated that specific eye motor programs could be linked to the neural processes that account for planning and control of different manual actions. Assuming this linkage, the authors interpreted their findings as evidence for a mirror neuron based (Gallese et al., 1996, Rizzolatti et al., 1996) based direct matching mechanism. Mapping the visual representation of an observed hand action onto a motor representation of the same action, this direct matching mechanism could indirectly allow the activation of eye movements, specifically linked to the motor representation of the hand action.
Thereby, the similarity between gaze behaviour in action execution and action observation could be explained.
As a further evidence for this interpretation, the authors accepted the absence of anticipatory eye movements in a control experiment with the actor hidden and the subject seeing only the moving object. For the activation of the mirror system, the presence of hand-object interaction is thought to be crucial (Gallese et al., 1996, Koski et al., 2002, Rizzolatti et al., 1996). Hence, in the control experiment the direct matching mechanism could not be stimulated sufficiently to evoke resonance in the motor representation during movement observation. A study on infants (Falck-Ytter, Gredeback, & von Hofsten, 2006) had reported similar findings already in 12 month olds, showing anticipatory eye movements during action observation, but only when a hand/object interaction was visible.
The aim of the present study was to investigate in more detail the dependency of anticipatory gaze behaviour on the visibility of hand-object interactions in the observed action. To this purpose a task similar to that used by Flanagan and Johansson (2003) was chosen, but with an indirect interaction between hand and manipulated object. Instead of real wooden blocks, objects were presented on a computer screen. They had to be grasped virtually by computer mouse’s click and moved by mouse movements to execute the stacking task. In addition, the importance of the nature of the actor was studied. Subjects either had to execute the task themselves (biological, self) or had to observe the experimenter (biological, other) or the computer (non-biological) doing the task.
Section snippets
Participants
The experiment was undertaken with the understanding and written consent of each recruited subject. Our study was accepted by the local ethics committee of the University of Ferrara and it was conforming to the Declaration of Helsinki (1983). Twenty-three right-handed volunteers (11 males, 12 females) between 20 and 35 years (mean 23.7 ± 3.4) participated in the study. All were students from the University of Ferrara, who had normal or corrected-to-normal vision (3 participants were wearing soft
Results
Mean PA and PT values, calculated over all the subjects in the different conditions, are shown in Fig. 2 together with the standard errors of mean. As indicated by the PA values, the subjects anticipated in both active conditions (SUB+/SUB−) on average over 50% of the total block movement time. However, in the passive conditions (EXP+/EXP−, HID+/HID−, COM+/COM−) subjects anticipated less than 50% of the block movement time. Interestingly, in no condition subjects anticipated less than 30% of
Discussion
All the subjects were anticipating to a considerable amount of time in the condition with self propelled block movements. This finding is in correspondence with various other experiments investigating gaze behaviour during active hand-object manipulation (Falck-Ytter et al., 2006, Flanagan and Johansson, 2003, Johansson et al., 2001, Land et al., 1999). In our experiment, an electronic mechanism transferred the effects of the hand movements on the computer screen. Hence, we demonstrate that
Conclusions
Our findings indicate that hand-object interactions are not the critical factor determining the occurrence of anticipatory eye movements. Indeed, no hand-object interactions were visible in our experiment. Nevertheless, anticipatory gaze behaviour was present during the observation of the effects of self-produced movements as well as during observation of the effects of movements produced by other individuals or even machines. In other words, anticipatory gaze behaviour does not simply depend
Acknowledgments
This research was supported by grants from MURST awarded to B.G and A.B. (Ministero dell’Università e della Ricerca Scientifica e Tecnologica, Italy) and by the European Commission grants MIRROR and ROBOTCUB.
References (13)
- et al.
Premotor cortex and the recognition of motor actions
Brain Research Cognitive Brain Research
(1996) - et al.
Memory representations in natural tasks
Journal of Cognitive Neuroscience
(1995) - et al.
Deictic codes for the embodiment of cognition
Behavioral and Brain Sciences
(1997) - et al.
Analysis of repeated measures
(1990) - et al.
Infants predict other people’s action goals
Nature Neuroscience
(2006) - et al.
Action plans used in action observation
Nature
(2003)
Cited by (26)
Gaze-based prediction of pen-based virtual interaction tasks
2015, International Journal of Human Computer StudiesCitation Excerpt :Alamargot et al. (2006) provide a detailed description of the eye movement characteristics recorded during reading and writing on a digital tablet. Gesierich et al. (2008) observe proactive (i.e. anticipatory) eye movements in both action execution and action observation during a two-user virtual block stacking task. Our work differs from the listed virtual task analyzers in its successful attempt at interpreting user behavior for online task prediction.
Eye movements during transitive action observation have sequential structure
2010, Acta PsychologicaCitation Excerpt :What is responsible for this characteristic sequence? One possibility is that it is a contingent result of the structure of the stimulus, plus the observer’s goal to establish the agent and patient (see e.g. Gesierich, Bruzzo, Ottoboni, & Finos, 2008). In our stimulus videos, the first detectable movement is of the agent, so it makes sense for the observer to identify the agent first. (
The Early Development of Object Knowledge: A Study of Infants' Visual Anticipations During Action Observation
2010, Developmental PsychologyTEACHER’S GAZE BLIND SPOT IN SCIENCE LECTURE CLASS
2023, Journal of Baltic Science EducationDesign Concepts for Gaze-Based Digital Musical Instruments
2022, Proceedings of the Sound and Music Computing ConferencesAction-related eye measures to assess surgical expertise
2021, BJS Open