Elsevier

Cognition

Volume 115, Issue 1, April 2010, Pages 46-53
Cognition

Creating number semantics through finger movement perception

https://doi.org/10.1016/j.cognition.2009.11.007Get rights and content

Abstract

Communication, language and conceptual knowledge related to concrete objects may rely on the sensory–motor systems from which they emerge. How abstract concepts can emerge from these systems is however still unknown. Here we report a functional interaction between a specific meaningful finger movement, such as a finger grip closing, and a concept as abstract as numerical magnitude. Participants were presented with Arabic digits to recall before or after they perceived a biological or non-biological hand movement. The results show that perceiving a grip closing slows down the processing of large magnitude numbers. Importantly, we show that this motor-to-semantic interaction differs from the reverse semantic-to-motor interaction, and that it does not result from a general movement amplitude processing as it is only observed for biological hand movements. These results demonstrate the functional link between number meaning and goal-directed finger movements, and show how abstract concept semantics can emerge from the sensory–motor circuits of the brain.

Introduction

How conceptual knowledge emerges, is represented in the brain, and exchanged with other members of the species has long been a matter of debate. After several decades dominated by the computational metaphor in which the nature of cognitive processes and conceptual knowledge was discussed independently from the brain and its functioning, the theory of embodied semantics proposed that concepts are built and represented within the same sensory–motor cerebral systems on which the enactment of these concepts rely (Gallese and Lakoff, 2005, Lakoff and Johnson, 1999, Martin, 2007). For example, the concept of “grasping” would be represented in the sensory–motor areas underlying grasping actions (Glover & Dixon, 2002). This theory is supported by neurophysiological data showing that “mirror” neurons originally discovered in the ventral premotor cortex (area F5c) of monkeys discharge not only when a monkey performs a motor action but also when it observes another monkey (or human being) performing the same action (Di Pellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992). Rostral to this mirror system lies a cortical area controlling the orofacial musculature and involved in communicative acts, that may be considered as the homologue of Broca’s area in the human brain (Petrides, Cadoret, and Mackey, (2005) for a discussion, see Toni, de Lange, Noordzij, and Hagoort (2008)). Together with this area, the mirror motor mechanism is viewed as the substrate on which communication and language evolve, for it represents the link between the sender and the receiver of a message (Arbib, 2005, Rizzolatti and Arbib, 1998). A similar motor resonance system exists in the human premotor cortex and constitutes the neural substrate of a corresponding mechanism recording the observed actions in the observer’s motor repertoire (Rizzolatti & Craighero, 2004). As a part of this system, the posterior parietal cortex is also involved in observing object-related actions; interestingly, observing an action mimicked without an object also induces a motor resonance in the premotor cortex (Buccino et al., 2001). Finally, it has been shown that observing a motor action fully engages the mirror system, especially the left premotor area, when the acting model is human but not when a non-biological actor performs the same action (Tai, Scherfler, Brooks, Sawamoto, & Castiello, 2004).

Sensory–motor networks could thus support conceptual knowledge building, which could give rise to semantic-to-motor interactions. Motor programs associated with objects are indeed accessed by language, and naming a feature of an object with a direct relation to a sensory–motor transformation can influence an action (Glover & Dixon, 2002). For example, the meaning of words printed on objects being reached for influences the kinematics of reaching and grasping movements (e.g., largest movements when the word “long” is printed on the object, irrespective of its real size; Gentilucci & Gangitano, 1998), processing action verbs impairs motor performance (e.g., by delaying the onset of concurrent reaching movements) whereas processing nouns does not (Boulenger et al., 2006), and processing action verbs but not concrete nouns is impaired in Parkinson’s disease (Boulenger et al., 2008). There is also an increasing body of findings showing that processing language triggers brain activity in sensory–motor areas (Aziz-Zadeh et al., 2006, Pulvermüller, 2005, Tettamanti et al., 2005). To date, semantic-to-motor interactions have mainly been described when processing images, words, sentences and concepts referring explicitly to concrete objects and actions or, at least, one of their parameters, and there have only been very few reports of implicit semantic knowledge of objects affecting hand movements (e.g., an increase in grip aperture following the presentation of words referring to large objects; Glover, Rosenbaum, Graham, & Dixon, 2004). That the sensory–motor system may be invoked for accessing or generating the semantics of words referring to actions (Hauk, Jonhsrude, & Pulvermüller, 2004) or manipulable objects (Almeida, Mahon, Nakayama, & Caramazza, 2008) is thus largely admitted, even by opponents of a general embodiment of cognition (e.g., Toni et al., 2008), but how it could account for abstract concept creation and communication remains highly controversial. Interestingly, semantic-to-motor interactions have been observed with the abstract concept of numerical magnitude. For example, processing the magnitude of Arabic numbers was found to interfere with finger movements, leading to compatibility effects when responses required a grip closing or opening: grip closing was initiated faster in response to small numbers, whereas grip opening was initiated faster in response to large numbers (Andres, Davare, Pesenti, Olivier, & Seron, 2004). Similar results have been reported for object-directed hand grips (Lindemann, Abolafia, Girardi, & Bekkering, 2007); in this case, the effect of number magnitude is higher during the first stages of object reaching movements and decreases as the hand gets closer to the object to grasp (Andres, Ostry, Nicol, & Paus, 2008). Recently, we showed that processing numerical information affected the judgment of actions toward objects, even when no actual action was required, but not a perceptual judgment per se (Badets, Andres, Di Luca, & Pesenti, 2007). Similarly, numerical magnitude was found to interact with an action judgment made on objects but not with a color judgment on the same objects (Chiou, Chang, Tzeng, & Wu, 2009).

So far, interactions have mainly been described from semantics to sensory–motor processes. From an evolutionary and developmental perspective, the functional link should, however, be in the other direction, from the sensory–motor system to semantics. Moreover, the previously reported numerical-to-motor interactions may emerge from a general magnitude system processing not only numbers but also any kind of magnitude (e.g., time and space; Dormal and Pesenti, 2007, Dormal et al., 2006, Walsh, 2003). In this study, we first established the effect of processing an abstract semantic concept on grip movement perception. We then reversed the interaction, and tested whether the perception of motor actions would influence the processing of an abstract semantic concept. We did this by assessing whether the perception of a movement mimicking an object-related action would influence number processing. We also tested the specificity of this putative functional link between the sensory–motor system and semantics by contrasting biological hand movements and identical non-biological fake hand movements. If number semantics truly relates to specific hand movements, we predict that motor-to-semantic interactions should only appear when a biological hand movement mimics a meaningful object-directed action.

To test these hypotheses, we designed two pairs of experiments. The first pair of experiments assessed semantic-to-motor interactions; the participants were presented with two Arabic digits to remember and had to recall one of them as soon as they perceived a hand movement. The second pair of experiments tested motor-to-semantic interactions; the participants had to name one of two Arabic digits after the presentation of hand movements. The hand movements were biological (Experiments 1 and 3) or non-biological (Experiments 2 and 4), and mimicked either a precision or a power grip closing or opening. Precision and power grips were chosen as they represent the main classes of grasping behavior in humans (Castiello, 2005, Napier, 1956, Susman, 1998). If space, numbers, and motor processes are represented within a general common magnitude system, interactions between hand-movement magnitude and number magnitude should be present in all experiments and conditions, irrespective of the nature of the movement (biological or fake hand), the type of grip (precision or power grip), and the type of object-directed action (opening or closing). Participants should answer faster to a small number after the presentation of a closing movement (because a closing movement would activate the meaning “small”), and to a large number after the presentation of an opening movement (because an opening movement would activate the meaning “large”). In contrast, if a common representation for number processing and biological movement perception is restricted to an object-related action, then interferences should only be observed for specific biological hand-movement conditions. When numerical magnitude is presented first, processing a large number should interfere with precision grip perception and processing a small number with power grip perception, as suggested by previous studies on semantic-to-motor interactions (Andres et al., 2004, Andres et al., 2008, Badets et al., 2007, Chiou et al., 2009; Lindemann et al., 2007). However, when movement perception comes first, both precision and power grips should interfere with large number processing but only when they mimic a closing movement, as observed during usual grasping, and not when they mimic an opening movement that is actually not related to object prehension; moreover, non-biological grip movements should never affect number processing.

Section snippets

General design

Two pairs of experiments were designed; the first pair (Experiments 1 and 2) tested semantic-to-motor interactions, and the second pair (Experiments 3 and 4) motor-to-semantic interactions. In Experiment 1, a pair of small or large Arabic digits, one odd and one even, was presented and followed by an imperative stimulus consisting of a finger movement mimicking an opening or closing movement. The participants had to recall the odd or the even number depending on the type (i.e., opening vs.

Global analysis

This analysis revealed the main effects of directions [semantic-to-motor: 568 ± 103 ms, motor-to-semantic: 521 ± 70 ms; F(1, 152) = 11.97, p < .001], hands [biological: 529 ± 73 ms, fake: 561 ± 105 ms; F(1, 152) = 5.54, p < .02], and magnitude [small: 542 ± 94 ms, large: 548 ± 95 ms; F(1, 152) = 5.26, p < .02]. Directions and hands interacted [F(1, 152) = 6.27, p < .01], with the latencies being significantly slower for the non-biological hands in the motor-to-semantic experiment compared to all the other conditions (about 75 ms

Discussion

The present study constitutes the first evidence that the perception of specific finger movements interferes with numerical magnitude processing. It shows that response latency increases when a grip closing preceded a response to a large but not a small number. This was observed with biological hands only. We suggest that this effect stems from the automatic activation of a code system sharing a dimension of magnitude across numbers, finger movements and, implicitly, objects, because fingers

Competing interests

The authors declare that they have no competing financial interests.

Acknowledgements

This study was supported by a Marie Curie Research Training Network from the European Community (MRTN-CT-2003-504927, Numbra project), and by Grant 01/06-267 from the Communauté Française de Belgique – Actions de Recherche Concertées (Belgium). We thank Nathalie Lefevre for her statistical advice. A.B. is a tenured researcher at the National Center for Scientific Research (France), and M.P. is a research associate at the National Fund for Scientific Research (Belgium).

References (46)

  • J. Almeida et al.

    Unconscious processing dissociates along categorical lines

    Proceedings of the National Academy of Sciences

    (2008)
  • M. Andres et al.

    Number magnitude and grip aperture interaction

    Neuroreport

    (2004)
  • M.A. Arbib

    From monkey-like action recognition to human language: An evolutionary framework for neurolinguistics

    Behavioral and Brain Sciences

    (2005)
  • A. Badets et al.

    Number magnitude potentiates action judgment

    Experimental Brain Research

    (2007)
  • Badets, A., Pesenti, M., & Olivier, E. (in press). Response–effect compatibility of finger–numeral configurations in...
  • F. Binkofski et al.

    A fronto-parietal circuit for object manipulation in man: Evidence from an fMRI-study

    European Journal of Neuroscience

    (1999)
  • V. Boulenger et al.

    Word processing in Parkinson’s disease is impaired for action verbs but not for concrete nouns

    Neuropsychologia

    (2008)
  • V. Boulenger et al.

    Cross-talk between language processes and overt motor behavior in the first 200 ms of processing

    Journal of Cognitive Neuroscience

    (2006)
  • G. Buccino et al.

    Action observation activates premotor and parietal areas in a somatotopic manner: An fMRI study

    European Journal of Neuroscience

    (2001)
  • U. Castiello

    The neuroscience of grasping

    Nature Reviews Neuroscience

    (2005)
  • R.Y.C. Chiou et al.

    The common magnitude code underlying numerical and size processing for action but not for perception

    Experimental Brain Research

    (2009)
  • J.D. Connolly et al.

    FMRI evidence for a “parietal reach region” in the human brain

    Experimental Brain Research

    (2003)
  • J.T. Coull et al.

    Where and when to pay attention: The neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI

    The Journal of Neuroscience

    (1998)
  • Cited by (66)

    • A common neural substrate for number comparison, hand reaching and grasping: A SDM-PSI meta-analysis of neuroimaging studies

      2022, Cortex
      Citation Excerpt :

      Secondly, we shed light on the issue of whether number and the hand action of reaching and grasping recruit overlapping brain areas (Aim 2). On the basis of previous behavioural studies showing the interplay between symbolic number processing and processes related to goal-directed hand actions (e.g., Andres et al., 2004; Andres et al., 2008; Badets et al., 2016; Badets & Pesenti, 2010, 2011; Ranzini et al., 2011), we embrace the idea of (partially) shared mechanisms underlying the hand action of reaching and grasping and number processing, and we argue that observing spatial overlap provides a stronger case for this hypothesis. In line with the ATOM theory (Walsh, 2003) and its recent extention (Cona, Weiner, Scarpazza, 2021), we predicted to observe such overlap primarily within the parietal lobes, in those areas which have been systematically involved in both the hand action of reaching and grasping and in number processing (e.g., IPS, PSPL).

    • The effect of multimodal information on children's numerical judgments

      2019, Journal of Experimental Child Psychology
      Citation Excerpt :

      This work demonstrates that young infants can track small sets cross-modally, whereas adults can make numerical judgments abstractly and independent of modality. However, developmental studies focusing on cross-modal numerical abilities during childhood have produced mixed results (Badets & Pesenti, 2010; Barth et al., 2005, 2007; Cote, 2015; Jordan & Baker, 2010; Mix et al., 1996). These studies have used critically different paradigms, resulting in noninterpretable findings leaving it unclear whether children’s numerical abilities are actually compromised in the context of cross-modal numerical stimuli or whether these performance costs are task specific.

    • Multi-dimensional sensorimotor grounding of concrete and abstract categories

      2023, Philosophical Transactions of the Royal Society B: Biological Sciences
    View all citing articles on Scopus
    View full text