Eye gaze reveals a fast, parallel extraction of the syntax of arithmetic formulas
Highlights
► Fixations organize in clusters tightly following the arithmetic hierarchical tree. ► Syntax in arithmetic is decoded in a glimpse even if constituted by procedurals rules. ► The duration of a fixation is predicted by the complexity of the syntactic level. ► Combining syntax and RTs to single operations predicts arithmetic formula RTs.
Introduction
Human language is characterized by a generative capacity: by embedding linguistic constituents inside each other, we can generate an infinite number of sentences. This capacity does not seem to exist in other animal communication systems (Chomsky, 1957). It has been suggested to arise from a core process of recursion which allows the formation of tree-like mental structures, not only in the field of language, but also in other cognitive domains such as music and mathematics (Hauser, Chomsky, & Fitch, 2002). Recursion would be a key distinctive feature separating humans from non-human primates.
While there is a vast corpus of evidence demonstrating a syntactic organization of language (Chomsky, 1988), and similar analyses of music (Lerdahl and Jackendoff, 1996, Patel, 2003), only a handful of studies have explored whether and how mathematical reasoning is organized at the syntactic level. These studies focused on the perception and recognition memory of mathematical expressions (Ernest, 1987, Nowak et al., 2000, Posner et al., 1984, Ranney, 1987). They demonstrated that syntactically well-formed substrings such as 4 − x can be more easily memorized than random non-grammatical strings such as x3 (Nowak et al., 2000). Providing further evidence in favor of a syntactic encoding of equations, Nowak et al. (2000) showed that within the well-formed sub-expressions contained in a string, those which form a phrasal node on the parse tree are more readily remembered. For instance, if the string 4 − x2(y + 7) is presented, the substring y + 7 is more likely to be memorized than the substring 4 − x2 (Nowak et al., 2000). In a recent study, Jansen, Marriott, and Yelland (2003) explored the scanning sequence of algebraic expressions using the Restricted Focus Viewer (RFV), whereas participants see at any time only a small region of the image in focus. The window can be moved using the computer mouse. As observed in language reading (Mitchell et al., 2008), symbols at the end of a phrasal constituent were scanned for significantly longer durations than symbols at the start or middle of the phrasal constituent (Nowak et al., 2000).
Landy and Goldstone, 2007a, Landy and Goldstone, 2007b have performed a series of experiments to measure the temporal and spatial allocation of attention in arithmetical expressions (Goldstone et al., 2010, Landy and Goldstone, 2007b, Landy et al., 2008). These experiments have shown that instead of memorizing that procedurally multiplication comes before addition, saliency maps may direct eye movements in such a way as to automatically instantiate this rule (Goldstone et al., 2010). In an experiment particularly relevant for the present study, Landy et al. (2008) measured eye-movements while subjects solve “multiplication–addition” problems (‘2 · 3 + 4’) and addition–multiplication (‘3 + 1 · 4’) problems. The results showed that saccades towards the multiplication sign tended to be earlier in the trial and to last longer than saccades to the addition sign (Landy et al., 2008).
In the present work we study how people solve arithmetic expressions with topologically equivalent parse trees, but with different spatial layouts such as 3 − (2 + (1 − 4)) or ((1 − 4) + 2) + 3. We contrast three alternative hypotheses on how syntax is processed in mathematical expressions, in relation to language. H1 Mathematical expressions are similar to language: an equation has to be scanned from left to right, sequentially, much like a sentence, while the nested syntactic structure is being computed. H2 The syntactic structure can be extracted very quickly and in parallel, but the typical left-to-right sequential organization of language is imprinted in mathematical thinking. This hypothesis predicts an initial bias which favors left to right parsing sequences. H3 The syntactic structure can be extracted in parallel, and language sets no bias on how mathematical expressions are read, predicting that all aspects of performance depend only on the topology of the parse-tree but are independent of their spatial layout.
We use eye movement patterns and chronometric measures to resolve these alternatives. Response-time data can distinguish H1 and H2 from H3 by demonstrating a cost for expressions whose parse tree does not have a left to right structure. Furthermore, hypotheses H1 and H2 establish distinct predictions on the precise trajectory of eye movements. H1 predicts a left-to right linear exploration in space. Instead, H2 predicts that only the initial fixations may reflect a reading bias, and very rapidly, the trajectory should reflect the hierarchical structure of the expression independently of its spatial layout.
Our data show strong evidence in favor of the second hypothesis: gaze is directed to the left of the equation, reflecting a bias which is consistent with language structuring, and then very rapidly follows the syntactic organization of the expression.
Section snippets
Participants
A total of 35 people participated in three independent experiments. All participants had finished high-school and thus had substantial educational practice in arithmetic. All the participants were native Spanish speakers. The experiment was approved by the local institutional ethics committee. Informed consent was obtained from all of the participants after the purpose and procedures of the experiment were fully explained.
Stimuli and procedure
Every trial started with the presentation of a fixation cross positioned
Participants
Thirteen people participated in the experiment 1 (mean age 24).
Stimuli and procedure
Arithmetic expressions were structured in a nested series of three operations delimited by parentheses and brackets. They belonged to two different classes: left branching “{(n1 ± n2) ± n3} ± n4” and right branching “n1 ± {n2 ± (n3 ± n4)}”. In all cases, the first operation to be performed according to syntactic structuring of the expression (referred as the deepest node of the hierarchy) was of the form (ni ± ni+1). In the left-branching
Experiment 2
Experiment 1 did not address whether the observed eye-movement patterns are driven by the abstract hierarchical structures of the equations ‘per se’ or merely by the presence of explicit visual cues. The equations were always hierarchically organized in terms of parentheses; with ‘round’ and ‘curly’ brackets further differentiating between the two nested levels of structural embedding. This raises the question of whether the same results would be obtained if instead of brackets, levels of
Experiment 3
Algebra and arithmetic have been considered a case study for the investigation of tasks involving nested sequences of operations where intermediate results are stored and reused in subsequent steps (Anderson & Lebiere, 1998). Multi-step human cognition has been modeled by applying the computer-science notion of “production system” (Anderson et al., 2004). It consists of a general mechanism for selecting production rules fuelled by sensory, motor, goal and memory modules. These models, as well
Main theoretical conclusion: extracting syntax at a glance
We showed that fixation sequences during arithmetic calculations are highly stereotyped and universal in a population of subjects with substantial mathematical training. The first fixation is directed slightly towards the left of the center of the expression. Trajectories then rapidly bifurcate in accordance with a hierarchical exploration of the syntactic tree. These trajectories are not affected by whether syntax is conveyed by brackets and parentheses, or by the precedence of arithmetic
Acknowledgements
This work was funded by the Human Frontiers Science Program. We thank Andrea Moro for reading the manuscript and for stimulating discussions on asymmetries on language and syntax.
References (56)
Cognitive arithmetic: A review of data and theory
Cognition
(1992)- et al.
Cultural recycling of cortical maps
Neuron
(2007) - et al.
Semantic evaluation of syntactic structure: Evidence from eye movements
Cognition
(2006) - et al.
The neural basis of perceptual learning
Neuron
(2001) - et al.
Reuniting perception and conception
Cognition
(1998) - et al.
Neuroimaging of syntax and syntactic processing
Current Opinion in Neurobiology
(2006) - et al.
An integrated theory of the mind
Psychological Review
(2004) - et al.
The atomic components of thought
(1998) - et al.
Mental addition: A test of three verification models
Memory and Cognition
(1981) - et al.
The neural reality of syntactic transformations: Evidence from functional magnetic resonance imaging
Psychological Science
(2003)