Elsevier

Cognition

Volume 125, Issue 3, December 2012, Pages 475-490
Cognition

Eye gaze reveals a fast, parallel extraction of the syntax of arithmetic formulas

https://doi.org/10.1016/j.cognition.2012.06.015Get rights and content

Abstract

Mathematics shares with language an essential reliance on the human capacity for recursion, permitting the generation of an infinite range of embedded expressions from a finite set of symbols. We studied the role of syntax in arithmetic thinking, a neglected component of numerical cognition, by examining eye movement sequences during the calculation of arithmetic expressions. Specifically, we investigated whether, similar to language, an expression has to be scanned sequentially while the nested syntactic structure is being computed or, alternatively, whether this structure can be extracted quickly and in parallel. Our data provide evidence for the latter: fixations sequences were stereotypically organized in clusters that reflected a fast identification of syntactic embeddings. A syntactically relevant pattern of eye movement was observed even when syntax was defined by implicit procedural rules (precedence of multiplication over addition) rather than explicit parentheses. While the total number of fixations was determined by syntax, the duration of each fixation varied with the complexity of the arithmetic operation at each step. These findings provide strong evidence for a syntactic organization for arithmetic thinking, paving the way for further comparative analysis of differences and coincidences in the instantiation of recursion in language and mathematics.

Highlights

► Fixations organize in clusters tightly following the arithmetic hierarchical tree. ► Syntax in arithmetic is decoded in a glimpse even if constituted by procedurals rules. ► The duration of a fixation is predicted by the complexity of the syntactic level. ► Combining syntax and RTs to single operations predicts arithmetic formula RTs.

Introduction

Human language is characterized by a generative capacity: by embedding linguistic constituents inside each other, we can generate an infinite number of sentences. This capacity does not seem to exist in other animal communication systems (Chomsky, 1957). It has been suggested to arise from a core process of recursion which allows the formation of tree-like mental structures, not only in the field of language, but also in other cognitive domains such as music and mathematics (Hauser, Chomsky, & Fitch, 2002). Recursion would be a key distinctive feature separating humans from non-human primates.

While there is a vast corpus of evidence demonstrating a syntactic organization of language (Chomsky, 1988), and similar analyses of music (Lerdahl and Jackendoff, 1996, Patel, 2003), only a handful of studies have explored whether and how mathematical reasoning is organized at the syntactic level. These studies focused on the perception and recognition memory of mathematical expressions (Ernest, 1987, Nowak et al., 2000, Posner et al., 1984, Ranney, 1987). They demonstrated that syntactically well-formed substrings such as 4  x can be more easily memorized than random non-grammatical strings such as x3 (Nowak et al., 2000). Providing further evidence in favor of a syntactic encoding of equations, Nowak et al. (2000) showed that within the well-formed sub-expressions contained in a string, those which form a phrasal node on the parse tree are more readily remembered. For instance, if the string 4  x2(y + 7) is presented, the substring y + 7 is more likely to be memorized than the substring 4  x2 (Nowak et al., 2000). In a recent study, Jansen, Marriott, and Yelland (2003) explored the scanning sequence of algebraic expressions using the Restricted Focus Viewer (RFV), whereas participants see at any time only a small region of the image in focus. The window can be moved using the computer mouse. As observed in language reading (Mitchell et al., 2008), symbols at the end of a phrasal constituent were scanned for significantly longer durations than symbols at the start or middle of the phrasal constituent (Nowak et al., 2000).

Landy and Goldstone, 2007a, Landy and Goldstone, 2007b have performed a series of experiments to measure the temporal and spatial allocation of attention in arithmetical expressions (Goldstone et al., 2010, Landy and Goldstone, 2007b, Landy et al., 2008). These experiments have shown that instead of memorizing that procedurally multiplication comes before addition, saliency maps may direct eye movements in such a way as to automatically instantiate this rule (Goldstone et al., 2010). In an experiment particularly relevant for the present study, Landy et al. (2008) measured eye-movements while subjects solve “multiplication–addition” problems (‘2 · 3 + 4’) and addition–multiplication (‘3 + 1 · 4’) problems. The results showed that saccades towards the multiplication sign tended to be earlier in the trial and to last longer than saccades to the addition sign (Landy et al., 2008).

In the present work we study how people solve arithmetic expressions with topologically equivalent parse trees, but with different spatial layouts such as 3  (2 + (1  4)) or ((1  4) + 2) + 3. We contrast three alternative hypotheses on how syntax is processed in mathematical expressions, in relation to language.

H1

Mathematical expressions are similar to language: an equation has to be scanned from left to right, sequentially, much like a sentence, while the nested syntactic structure is being computed.

H2

The syntactic structure can be extracted very quickly and in parallel, but the typical left-to-right sequential organization of language is imprinted in mathematical thinking. This hypothesis predicts an initial bias which favors left to right parsing sequences.

H3

The syntactic structure can be extracted in parallel, and language sets no bias on how mathematical expressions are read, predicting that all aspects of performance depend only on the topology of the parse-tree but are independent of their spatial layout.

We use eye movement patterns and chronometric measures to resolve these alternatives. Response-time data can distinguish H1 and H2 from H3 by demonstrating a cost for expressions whose parse tree does not have a left to right structure. Furthermore, hypotheses H1 and H2 establish distinct predictions on the precise trajectory of eye movements. H1 predicts a left-to right linear exploration in space. Instead, H2 predicts that only the initial fixations may reflect a reading bias, and very rapidly, the trajectory should reflect the hierarchical structure of the expression independently of its spatial layout.

Our data show strong evidence in favor of the second hypothesis: gaze is directed to the left of the equation, reflecting a bias which is consistent with language structuring, and then very rapidly follows the syntactic organization of the expression.

Section snippets

Participants

A total of 35 people participated in three independent experiments. All participants had finished high-school and thus had substantial educational practice in arithmetic. All the participants were native Spanish speakers. The experiment was approved by the local institutional ethics committee. Informed consent was obtained from all of the participants after the purpose and procedures of the experiment were fully explained.

Stimuli and procedure

Every trial started with the presentation of a fixation cross positioned

Participants

Thirteen people participated in the experiment 1 (mean age 24).

Stimuli and procedure

Arithmetic expressions were structured in a nested series of three operations delimited by parentheses and brackets. They belonged to two different classes: left branching “{(n1 ± n2) ± n3} ± n4” and right branching “n1 ± {n2 ± (n3 ± n4)}”. In all cases, the first operation to be performed according to syntactic structuring of the expression (referred as the deepest node of the hierarchy) was of the form (ni ± ni+1). In the left-branching

Experiment 2

Experiment 1 did not address whether the observed eye-movement patterns are driven by the abstract hierarchical structures of the equations ‘per se’ or merely by the presence of explicit visual cues. The equations were always hierarchically organized in terms of parentheses; with ‘round’ and ‘curly’ brackets further differentiating between the two nested levels of structural embedding. This raises the question of whether the same results would be obtained if instead of brackets, levels of

Experiment 3

Algebra and arithmetic have been considered a case study for the investigation of tasks involving nested sequences of operations where intermediate results are stored and reused in subsequent steps (Anderson & Lebiere, 1998). Multi-step human cognition has been modeled by applying the computer-science notion of “production system” (Anderson et al., 2004). It consists of a general mechanism for selecting production rules fuelled by sensory, motor, goal and memory modules. These models, as well

Main theoretical conclusion: extracting syntax at a glance

We showed that fixation sequences during arithmetic calculations are highly stereotyped and universal in a population of subjects with substantial mathematical training. The first fixation is directed slightly towards the left of the center of the expression. Trajectories then rapidly bifurcate in accordance with a hierarchical exploration of the syntactic tree. These trajectories are not affected by whether syntax is conveyed by brackets and parentheses, or by the precedence of arithmetic

Acknowledgements

This work was funded by the Human Frontiers Science Program. We thank Andrea Moro for reading the manuscript and for stimulating discussions on asymmetries on language and syntax.

References (56)

  • K.S. Binder et al.

    Extraction of information to the left of the fixated word in reading

    Journal of Experimental Psychology: Human Perception and Performance

    (1999)
  • F. Cajori

    A history of mathematical notations

    (1929)
  • N. Chomsky

    Sytactic structures

    (1957)
  • N. Chomsky

    Reflections on language

    (1975)
  • N. Chomsky

    Language and problems of language

    (1988)
  • N. Chomsky

    The minimalist program

    (1995)
  • P. Ernest

    A model of the cognitive meaning of mathematical expressions

    British Journal of Educational Psychology

    (1987)
  • J.A. Fodor

    The language of thought

    (1975)
  • R.L. Goldstone et al.

    The education of perception

    Topics in Cognitive Science

    (2010)
  • Grodzinsky, Y. (2000). The neurology of syntax: Language use without Broca’s area. Behavioral and Brain Sciences,...
  • Y. Grodzinsky et al.

    The neurology of empty categories aphasics’ failure to detect ungrammaticality

    Journal of Cognitive Neuroscience

    (1998)
  • M.D. Hauser et al.

    The faculty of language: What is it, who has it, and how did it evolve?

    Science

    (2002)
  • A.R. Jansen et al.

    Comprehension of algebraic expressions by experienced users of mathematics

    Quarterly Journal of Experimental Psychology: A, Human Experimental Psychology

    (2003)
  • D. Kirshner

    The visual syntax of algebra

    Journal for Research in Mathematics Education

    (1989)
  • R. Kliegl

    Toward a perceptual-span theory of distributed processing in reading: A reply to Rayner, Pollatsek, Drieghe, Slattery, and Reichle (2007)

    Journal of Experimental Psychology: General

    (2007)
  • R. Kliegl et al.

    Tracking the mind during reading: The influence of past, present, and future words on fixation durations

    Journal of Experimental Psychology: General

    (2006)
  • K. Kofka

    Principles of Gestalt psychology

    (1935)
  • D. Landy et al.

    Formal notations are diagrams: Evidence from a production task

    Memory and Cognition

    (2007)
  • Cited by (0)

    View full text