Elsevier

Cognition

Volume 136, March 2015, Pages 228-246
Cognition

Eye movements reveal memory processes during similarity- and rule-based decision making

https://doi.org/10.1016/j.cognition.2014.11.019Get rights and content

Highlights

  • We study differences in similarity- and rule-based processes using eye movements.

  • Using rule- and similarity-based strategies induces different gaze patterns.

  • Gaze patterns show that exemplars are retrieved from memory when using similarity.

  • No exemplars are retrieved when using a rule-based process.

  • Eye movements can be used to study memory processes in decision making.

Abstract

Recent research suggests that when people retrieve information from memory they tend to fixate on the location where the information had appeared during encoding. We used this phenomenon to investigate if different information is activated in memory when people use a rule- versus a similarity-based decision strategy. In two studies, participants first memorized multiple pieces of information about various job candidates (exemplars). In subsequent test trials they judged the suitability of new candidates that varied in their similarity to the previously learned exemplars. Results show that when using similarity, but not when using a rule, participants fixated longer on the previous location of exemplars that resembled the new candidates than on the location of dissimilar exemplars. This suggests that people using similarity retrieve previously learned exemplars, whereas people using a rule do not. The study illustrates that eye movements can provide new insights into the memory processes underlying decision making.

Introduction

A fundamental distinction in cognitive psychology refers to the contrast between similarity- and rule-based cognitive processes. Although this distinction is intuitively appealing and has stimulated much empirical research, it has proved difficult to pin down on the process level (e.g., Barsalou, 1990, Hahn and Chater, 1998, Milton et al., 2009, Pothos, 2005). One reason could be that a core difference between rule-based and similarity-based processes lies in how information is processed in memory (Hahn & Chater, 1998). This makes the differences between similarity- and rule-based processes difficult to study, because memory processes are hard to observe. For instance, when studying decision processes it is easy to observe what people chose, but not whether people made a choice by focusing on the information provided or by retrieving similar decisions from memory. Recent research has suggested that eye movements can be used to trace information search in memory (Jahn and Braatz, 2014, Renkewitz and Jahn, 2010, Renkewitz and Jahn, 2012, Richardson and Kirkham, 2004, Richardson and Spivey, 2000). We show in the present work that recording eye movements can be used to make differences in memory retrieval between people using similarity- and rule-based strategies visible, providing a possible method for disentangling the two strategies on the process level.

Studying cognitive processes that rely on memory, such as categorization, reasoning, problem solving, and decision making, can be challenging because the processes of interest are not directly observable. Researchers have tackled this problem by developing indirect methods, using self-reports, computational modeling, and reaction times to gain a window into the mind (Anderson, 1987, Bröder, 2000, Johnson and Krems, 2001, Lewandowsky and Farrell, 2011, Mehlhorn et al., 2011, Payne et al., 1993). Although these methods provide valuable data, they also have important drawbacks. For instance, self-reports about memory processes are often inaccurate and incomplete, and asking about them can affect the process itself (Ericsson and Simon, 1980, Renkewitz and Jahn, 2010, Russo et al., 1989).

Alternatively, eye movements can be used to trace information search (Glaholt and Reingold, 2011, Orquin and Mueller Loose, 2013, Peterson and Beck, 2011). Eye movements are quick, frequent, and highly automatic actions (Irwin, 2004, Rayner, 2009, Spivey and Dale, 2011, van Gompel et al., 2007) that have been shown to reflect attention and information search in a variety of tasks, such as concept learning (Nelson and Cottrell, 2007, Rehder and Hoffman, 2005), text comprehension (Allopenna et al., 1998, Altmann, 2004, Altmann and Kamide, 2007, Tanenhaus et al., 1995), and decision making (Glaholt and Reingold, 2011, Orquin and Mueller Loose, 2013). Lately, evidence has been accumulating that eye movements can also be used to trace memory processes. When people retrieve information from memory they look at spatial locations where the information was originally presented—even if the information is no longer visible (Hoover and Richardson, 2008, Johansson et al., 2012, Johansson et al., 2006, Laeng et al., 2014, Laeng and Teodorescu, 2002, Martarelli and Mast, 2013, Richardson and Kirkham, 2004, Richardson and Spivey, 2000, Spivey and Geng, 2001). In the classic paradigm, Richardson and Spivey (2000) presented participants with a spinning cross in one of four equal-sized areas on a computer screen together with spoken factual information. In a later test phase, participants heard a statement regarding the presented facts and had to judge the truth of the statement. Even though during this retrieval phase the computer screen was blank, participants fixated more often on the spatial area where the sought-after information had been presented than on the other three areas on the screen.

Most likely, people show this “looking at nothing” effect because during encoding, information from multiple sources of input, including the locations of perceived objects, is integrated into an episodic memory representation. Once the episodic memory representation is reactivated during retrieval it spreads activation to the motor system, which in turn leads to the execution of eye movements back to the locations linked with the memory representation (Huettig et al., 2012, Huettig et al., 2011, Richardson and Kirkham, 2004). The exact role eye movements play in the retrieval process is still debated (e.g., Ferreira et al., 2008, Richardson et al., 2009), but early evidence suggests that eye movements can also facilitate memory retrieval (Johansson and Johansson, 2014, Laeng et al., 2014, Scholz et al., in press).

Recent research suggests that the looking-at-nothing effect can also be used to trace retrieval processes in higher order cognitive processes such as decision making and diagnostic reasoning. For instance, Renkewitz and Jahn, 2010, Renkewitz and Jahn, 2012 found that when participants had to retrieve information about two alternatives to make a decision, they looked at the location where the information about the alternatives had previously appeared. Furthermore, gaze patterns during retrieval were consistent with the information search predicted by the decision strategies participants used. Similarly, Jahn and Braatz (2014) showed that during a diagnostic reasoning task, people tended to look at locations associated with symptoms they had to retrieve from memory to test hypotheses about what caused the symptom. More importantly, the eye movements reflected the diagnostic value of the symptoms and how participants updated their hypotheses about the causes over time. These findings suggest that eye movements are not automatically launched to all associated spatial locations but reflect target-oriented information search in memory during the reasoning process.

In sum, spatial information about the location of information is stored along with the memory of it. Retrieving the respective memory triggers eye movements to the associated locations. These eye movements reflect the currently active memory representation and provide researchers with a new method for monitoring information search in memory. We used this method to differentiate memory processes involved in similarity- and rule-based judgments and decisions.

The distinction between rule- and similarity-based processes is fundamental to understanding human cognition and has stimulated research in a broad range of fields, from categorization and decision making (e.g., Ashby et al., 1998, Erickson et al., 1998, Persson and Rieskamp, 2009, Pothos and Hahn, 2000) to reasoning (Smith, Langston, & Nisbett, 1992) and language acquisition (Pinker & Prince, 1988). In general, it is assumed that rule-based processes involve the application of previously abstracted knowledge to specific instances (Hahn & Chater, 1998). That is, people form a rule defining the relationship between a specific piece of information and the decision outcome and apply it when confronted with a new decision problem (Bröder et al., 2010, Juslin et al., 2008, Mata et al., 2012, Persson and Rieskamp, 2009, von Helversen et al., 2010, von Helversen and Rieskamp, 2008, von Helversen and Rieskamp, 2009). For instance, when deciding to take one’s bike or car in the morning, one could have learned the rule that it is better to take the car when it is raining. In contrast, similarity processes are generally characterized by the retrieval of similar instances or exemplars from memory (Bröder et al., 2010, Hahn and Chater, 1998, Hahn et al., 2010, Juslin and Persson, 2002). That is, when deciding to take the car or the bike in the morning, one might think back to similar occasions and compare how well one fared when taking the bike.

A core theoretical distinction that has been proposed is that the two processes differ in the way mental representations of stored information are accessed (Bailey, 2005, Hahn and Chater, 1998). Similarity-based processes involve comparing the object under consideration to exemplars stored in memory. In contrast, rule-based processes involve processing the information an object under consideration provides according to the processing steps specified by the rule. Accordingly, in a decision task the object’s attributes are matched against the conditions for choosing the respective options as specified in the rule. This suggests that similarity-based but not rule-based processes require the retrieval of previously encountered instances from memory. Consistently, similarity-based judgments rely more on episodic memory than rule-based judgments (Hoffmann, von Helversen, & Rieskamp, 2014). However, direct evidence that similarity- and rule-based processes rely on different retrieval processes is scarce (Ashby & O’Brien, 2005). One problem is that differentiating the two processes is far from trivial on a conceptual and empirical level (Barsalou, 1990, Hahn and Chater, 1998, Markman et al., 2005, Pothos, 2005). Research trying to tease apart rule- and similarity-based processes has frequently relied on computational modeling approaches (e.g., Bröder et al., 2010, Juslin et al., 2008, Juslin et al., 2003, Karlsson et al., 2007, Nosofsky and Bergert, 2007, Pachur and Olsson, 2012, Persson and Rieskamp, 2009, Platzer and Bröder, 2013, von Helversen et al., 2013, von Helversen et al., 2010). Although computational modeling approaches can provide relevant insights into the cognitive processes underlying behavior, there are important limitations. First, the decision of which model best describes the data is usually based on some measure of goodness of fit. However, depending on the selected measure the results may diverge considerably (Scheibehenne, Rieskamp, & Wagenmakers, 2013). Furthermore, just because a model can predict the outcome of a decision process does not necessarily mean it also reflects the underlying cognitive processes. Indeed, looking at process data may reveal that a model misses important aspects of the cognitive processes leading to the decision (e.g., Johnson, Schulte-Mecklenbeck, & Willemsen, 2008). Accordingly, it seems necessary to complement cognitive modeling approaches with process data to reach a full understanding of the cognitive processes underlying a decision (see also Schulte-Mecklenbeck, Kühberger, & Ranyard, 2011).

We used the looking-at-nothing effect to clarify how memory processes involved in similarity- and rule-based decisions differ. Specifically, if rule and similarity processes differ in the information that is retrieved from memory when making a decision, it should be possible to make these search processes visible by associating exemplars with specific spatial locations and then tracking the eye movements during the retrieval process to capture information search in memory. If people retrieve exemplars from memory when relying on a similarity-based process, the looking-at-nothing effect would predict that people gaze back at associated exemplar locations. In contrast, if people do not retrieve similar exemplars from memory when using a rule, fixation on the locations associated with exemplars should be rare. Furthermore, when using an exemplar-based strategy the eye movements to exemplar locations should be a function of the exemplars’ similarity, because the probability with which an exemplar is retrieved from memory depends on the exemplar’s similarity to the object under evaluation (Dougherty et al., 1999, Hintzman, 1988, Nosofsky and Palmeri, 1997).

To test these hypotheses, we conducted two experiments using a multi-cue decision paradigm. We chose this type of problem because the assumption that people rely on rule- and similarity-based strategies to make decisions is widespread (Bröder et al., 2010, Hahn et al., 2010, Juslin et al., 2003, Juslin et al., 2008, Karlsson et al., 2007, Pachur and Olsson, 2012, Persson and Rieskamp, 2009, Platzer and Bröder, 2013, von Helversen et al., 2010, von Helversen et al., 2013).

Section snippets

Study 1

Study 1 examined if relying on a rule versus relying on similarity leads to different information retrieval from memory, which, in turn, is reflected in different eye movements. Participants had to decide if job candidates applying for a position were suitable, that is, whether they should be invited for an interview or rejected. In a training phase participants learned information about two suitable and two unsuitable job candidates. In a subsequent test phase they were instructed to decide if

Study 2

The purpose of Study 2 was to test if information retrieval from memory also differs between rule-based and similarity-based decision strategies when the strategy is employed spontaneously. To be able to compare explicit and spontaneous strategy use we investigated the eye movements related to memory processes when strategies are spontaneously employed and when explicit instructions are given to use a specific strategy.

Research in categorization, judgment, and decision making based on cognitive

General discussion

When making everyday decisions from memory people can apply abstract rules that process the available information for a decision or they can make a decision according to similar decision situations encountered in the past (Ashby et al., 1998, Erickson et al., 1998, Juslin and Persson, 2002, Nosofsky et al., 1994, Platzer and Bröder, 2013). Although this distinction is intuitively appealing it proves hard to separate on a process level. One reason is that the two processes are conceptually

Conclusion

By observing eye movements while people performed memory-based decisions using a similarity-based or a rule-based strategy, we showed that the two strategies involve different memory processes. Although similarity and rule users had built the same memory representations, they differed in how these representations were accessed when making a decision. Whereas similarity users retrieved information about similar exemplars, rule users did not—providing empirical evidence that the two processes can

Acknowledgments

The authors would like to thank Claudia Dietzel, Jet Hoe Tang, Regina Weilbächer, Stefan Thommen, Nancy John, and Christian Amstad for their help in conducting the experiments and Josef Krems for encouraging this research. The research was supported by Grant D/12/43949 from the German Academic Exchange Service to the first author and Swiss National Science Foundation research Grants 100014_130192 and 100014_146169 to the second and third author.

References (85)

  • P. Juslin et al.

    Information integration in multiple cue judgment: A division of labor hypothesis

    Cognition

    (2008)
  • P. Juslin et al.

    PROBabilities from EXemplars (PROBEX): A “lazy” algorithm for probabilistic inference from generic knowledge

    Cognitive Science

    (2002)
  • B. Laeng et al.

    Scrutinizing visual images: The role of gaze in mental imagery and memory

    Cognition

    (2014)
  • B. Laeng et al.

    Eye scanpaths during visual imagery reenact those of perception of the same visual scene

    Cognitive Science

    (2002)
  • F. Milton et al.

    The neural basis of overall similarity and single-dimension sorting

    NeuroImage

    (2009)
  • J.D. Nelson et al.

    A probabilistic model of eye movements in concept formation

    Neurocomputing

    (2007)
  • J.L. Orquin et al.

    Attention and choice: A review on eye movements in decision making

    Acta Psychologica

    (2013)
  • T. Pachur et al.

    Type of learning task impacts performance and strategy selection in decision making

    Cognitive Psychology

    (2012)
  • M. Persson et al.

    Inferences from memory: Strategy- and exemplar-based judgment models compared

    Acta Psychologica

    (2009)
  • S. Pinker et al.

    On language processing and connectionism: Analysis of a parallel distributed processing model of language acquisition

    Cognition

    (1988)
  • B. Rehder et al.

    Eyetracking and selective attention in category learning

    Cognitive Psychology

    (2005)
  • D.C. Richardson et al.

    Much ado about eye movements to nothing: A response to Ferreira et al.: Taking a new look at looking at nothing

    Trends in Cognitive Sciences

    (2009)
  • D.C. Richardson et al.

    Representation, space and Hollywood Squares: Looking at things that aren’t there anymore

    Cognition

    (2000)
  • E.E. Smith et al.

    The case for rules in reasoning

    Cognitive Science

    (1992)
  • B. von Helversen et al.

    Why does cue polarity information provide benefits in inference problems? The role of strategy selection and knowledge of cue importance

    Acta Psychologica

    (2013)
  • G.T.M. Altmann

    Language-mediated eye movements in the absence of a visual world: The “blank screen paradigm”

    Cognition

    (2004)
  • J.R. Anderson

    Methodologies for studying human knowledge

    Behavioral and Brain Sciences

    (1987)
  • F.G. Ashby et al.

    A neuropsychological theory of multiple systems in category learning

    Psychological Review

    (1998)
  • F.G. Ashby et al.

    Human category learning

    Annual Review of Psychology

    (2005)
  • T.M. Bailey

    Rules work on one representation; similarity compares two representations

    Behavioral and Brain Sciences

    (2005)
  • L.W. Barsalou

    On the indistinguishability of exemplar memory and abstraction in category representation

  • A. Bröder

    A methodological comment on behavioral decision making

    Psychologische Beiträge

    (2000)
  • A. Bröder et al.

    Cue integration vs. exemplar-based reasoning in multi-attribute decisions from memory: A matter of cue representation

    Judgment and Decision Making

    (2010)
  • L.R. Brooks et al.

    Instantiated features and the use of “rules”

    Journal of Experimental Psychology: General

    (2006)
  • M.R.P. Dougherty et al.

    MINERVA-DM: A memory process model for judgments of likelihood

    Psychological Review

    (1999)
  • M.A. Erickson et al.

    Rules and exemplars in category learning

    Journal of Experimental Psychology: General

    (1998)
  • K.A. Ericsson et al.

    Verbal reports as data

    Psychological Review

    (1980)
  • M.G. Glaholt et al.

    Eye movement monitoring as a process tracing methodology in decision making research

    Journal of Neuroscience, Psychology, and Economics

    (2011)
  • C. Harries et al.

    Measuring doctors’ self-insight into their treatment decisions

    Applied Cognitive Psychology

    (2000)
  • D.L. Hintzman

    Judgments of frequency and recognition memory in a multiple-trace memory model

    Psychological Review

    (1988)
  • J.A. Hoffmann et al.

    Deliberation’s blindsight: How cognitive load can improve judgments

    Psychological Science

    (2013)
  • J.A. Hoffmann et al.

    Pillars of judgment: How memory abilities affect performance in rule-based and exemplar-based judgments

    Journal of Experimental Psychology: General

    (2014)
  • Cited by (0)

    View full text