Eye movements reveal memory processes during similarity- and rule-based decision making
Introduction
A fundamental distinction in cognitive psychology refers to the contrast between similarity- and rule-based cognitive processes. Although this distinction is intuitively appealing and has stimulated much empirical research, it has proved difficult to pin down on the process level (e.g., Barsalou, 1990, Hahn and Chater, 1998, Milton et al., 2009, Pothos, 2005). One reason could be that a core difference between rule-based and similarity-based processes lies in how information is processed in memory (Hahn & Chater, 1998). This makes the differences between similarity- and rule-based processes difficult to study, because memory processes are hard to observe. For instance, when studying decision processes it is easy to observe what people chose, but not whether people made a choice by focusing on the information provided or by retrieving similar decisions from memory. Recent research has suggested that eye movements can be used to trace information search in memory (Jahn and Braatz, 2014, Renkewitz and Jahn, 2010, Renkewitz and Jahn, 2012, Richardson and Kirkham, 2004, Richardson and Spivey, 2000). We show in the present work that recording eye movements can be used to make differences in memory retrieval between people using similarity- and rule-based strategies visible, providing a possible method for disentangling the two strategies on the process level.
Studying cognitive processes that rely on memory, such as categorization, reasoning, problem solving, and decision making, can be challenging because the processes of interest are not directly observable. Researchers have tackled this problem by developing indirect methods, using self-reports, computational modeling, and reaction times to gain a window into the mind (Anderson, 1987, Bröder, 2000, Johnson and Krems, 2001, Lewandowsky and Farrell, 2011, Mehlhorn et al., 2011, Payne et al., 1993). Although these methods provide valuable data, they also have important drawbacks. For instance, self-reports about memory processes are often inaccurate and incomplete, and asking about them can affect the process itself (Ericsson and Simon, 1980, Renkewitz and Jahn, 2010, Russo et al., 1989).
Alternatively, eye movements can be used to trace information search (Glaholt and Reingold, 2011, Orquin and Mueller Loose, 2013, Peterson and Beck, 2011). Eye movements are quick, frequent, and highly automatic actions (Irwin, 2004, Rayner, 2009, Spivey and Dale, 2011, van Gompel et al., 2007) that have been shown to reflect attention and information search in a variety of tasks, such as concept learning (Nelson and Cottrell, 2007, Rehder and Hoffman, 2005), text comprehension (Allopenna et al., 1998, Altmann, 2004, Altmann and Kamide, 2007, Tanenhaus et al., 1995), and decision making (Glaholt and Reingold, 2011, Orquin and Mueller Loose, 2013). Lately, evidence has been accumulating that eye movements can also be used to trace memory processes. When people retrieve information from memory they look at spatial locations where the information was originally presented—even if the information is no longer visible (Hoover and Richardson, 2008, Johansson et al., 2012, Johansson et al., 2006, Laeng et al., 2014, Laeng and Teodorescu, 2002, Martarelli and Mast, 2013, Richardson and Kirkham, 2004, Richardson and Spivey, 2000, Spivey and Geng, 2001). In the classic paradigm, Richardson and Spivey (2000) presented participants with a spinning cross in one of four equal-sized areas on a computer screen together with spoken factual information. In a later test phase, participants heard a statement regarding the presented facts and had to judge the truth of the statement. Even though during this retrieval phase the computer screen was blank, participants fixated more often on the spatial area where the sought-after information had been presented than on the other three areas on the screen.
Most likely, people show this “looking at nothing” effect because during encoding, information from multiple sources of input, including the locations of perceived objects, is integrated into an episodic memory representation. Once the episodic memory representation is reactivated during retrieval it spreads activation to the motor system, which in turn leads to the execution of eye movements back to the locations linked with the memory representation (Huettig et al., 2012, Huettig et al., 2011, Richardson and Kirkham, 2004). The exact role eye movements play in the retrieval process is still debated (e.g., Ferreira et al., 2008, Richardson et al., 2009), but early evidence suggests that eye movements can also facilitate memory retrieval (Johansson and Johansson, 2014, Laeng et al., 2014, Scholz et al., in press).
Recent research suggests that the looking-at-nothing effect can also be used to trace retrieval processes in higher order cognitive processes such as decision making and diagnostic reasoning. For instance, Renkewitz and Jahn, 2010, Renkewitz and Jahn, 2012 found that when participants had to retrieve information about two alternatives to make a decision, they looked at the location where the information about the alternatives had previously appeared. Furthermore, gaze patterns during retrieval were consistent with the information search predicted by the decision strategies participants used. Similarly, Jahn and Braatz (2014) showed that during a diagnostic reasoning task, people tended to look at locations associated with symptoms they had to retrieve from memory to test hypotheses about what caused the symptom. More importantly, the eye movements reflected the diagnostic value of the symptoms and how participants updated their hypotheses about the causes over time. These findings suggest that eye movements are not automatically launched to all associated spatial locations but reflect target-oriented information search in memory during the reasoning process.
In sum, spatial information about the location of information is stored along with the memory of it. Retrieving the respective memory triggers eye movements to the associated locations. These eye movements reflect the currently active memory representation and provide researchers with a new method for monitoring information search in memory. We used this method to differentiate memory processes involved in similarity- and rule-based judgments and decisions.
The distinction between rule- and similarity-based processes is fundamental to understanding human cognition and has stimulated research in a broad range of fields, from categorization and decision making (e.g., Ashby et al., 1998, Erickson et al., 1998, Persson and Rieskamp, 2009, Pothos and Hahn, 2000) to reasoning (Smith, Langston, & Nisbett, 1992) and language acquisition (Pinker & Prince, 1988). In general, it is assumed that rule-based processes involve the application of previously abstracted knowledge to specific instances (Hahn & Chater, 1998). That is, people form a rule defining the relationship between a specific piece of information and the decision outcome and apply it when confronted with a new decision problem (Bröder et al., 2010, Juslin et al., 2008, Mata et al., 2012, Persson and Rieskamp, 2009, von Helversen et al., 2010, von Helversen and Rieskamp, 2008, von Helversen and Rieskamp, 2009). For instance, when deciding to take one’s bike or car in the morning, one could have learned the rule that it is better to take the car when it is raining. In contrast, similarity processes are generally characterized by the retrieval of similar instances or exemplars from memory (Bröder et al., 2010, Hahn and Chater, 1998, Hahn et al., 2010, Juslin and Persson, 2002). That is, when deciding to take the car or the bike in the morning, one might think back to similar occasions and compare how well one fared when taking the bike.
A core theoretical distinction that has been proposed is that the two processes differ in the way mental representations of stored information are accessed (Bailey, 2005, Hahn and Chater, 1998). Similarity-based processes involve comparing the object under consideration to exemplars stored in memory. In contrast, rule-based processes involve processing the information an object under consideration provides according to the processing steps specified by the rule. Accordingly, in a decision task the object’s attributes are matched against the conditions for choosing the respective options as specified in the rule. This suggests that similarity-based but not rule-based processes require the retrieval of previously encountered instances from memory. Consistently, similarity-based judgments rely more on episodic memory than rule-based judgments (Hoffmann, von Helversen, & Rieskamp, 2014). However, direct evidence that similarity- and rule-based processes rely on different retrieval processes is scarce (Ashby & O’Brien, 2005). One problem is that differentiating the two processes is far from trivial on a conceptual and empirical level (Barsalou, 1990, Hahn and Chater, 1998, Markman et al., 2005, Pothos, 2005). Research trying to tease apart rule- and similarity-based processes has frequently relied on computational modeling approaches (e.g., Bröder et al., 2010, Juslin et al., 2008, Juslin et al., 2003, Karlsson et al., 2007, Nosofsky and Bergert, 2007, Pachur and Olsson, 2012, Persson and Rieskamp, 2009, Platzer and Bröder, 2013, von Helversen et al., 2013, von Helversen et al., 2010). Although computational modeling approaches can provide relevant insights into the cognitive processes underlying behavior, there are important limitations. First, the decision of which model best describes the data is usually based on some measure of goodness of fit. However, depending on the selected measure the results may diverge considerably (Scheibehenne, Rieskamp, & Wagenmakers, 2013). Furthermore, just because a model can predict the outcome of a decision process does not necessarily mean it also reflects the underlying cognitive processes. Indeed, looking at process data may reveal that a model misses important aspects of the cognitive processes leading to the decision (e.g., Johnson, Schulte-Mecklenbeck, & Willemsen, 2008). Accordingly, it seems necessary to complement cognitive modeling approaches with process data to reach a full understanding of the cognitive processes underlying a decision (see also Schulte-Mecklenbeck, Kühberger, & Ranyard, 2011).
We used the looking-at-nothing effect to clarify how memory processes involved in similarity- and rule-based decisions differ. Specifically, if rule and similarity processes differ in the information that is retrieved from memory when making a decision, it should be possible to make these search processes visible by associating exemplars with specific spatial locations and then tracking the eye movements during the retrieval process to capture information search in memory. If people retrieve exemplars from memory when relying on a similarity-based process, the looking-at-nothing effect would predict that people gaze back at associated exemplar locations. In contrast, if people do not retrieve similar exemplars from memory when using a rule, fixation on the locations associated with exemplars should be rare. Furthermore, when using an exemplar-based strategy the eye movements to exemplar locations should be a function of the exemplars’ similarity, because the probability with which an exemplar is retrieved from memory depends on the exemplar’s similarity to the object under evaluation (Dougherty et al., 1999, Hintzman, 1988, Nosofsky and Palmeri, 1997).
To test these hypotheses, we conducted two experiments using a multi-cue decision paradigm. We chose this type of problem because the assumption that people rely on rule- and similarity-based strategies to make decisions is widespread (Bröder et al., 2010, Hahn et al., 2010, Juslin et al., 2003, Juslin et al., 2008, Karlsson et al., 2007, Pachur and Olsson, 2012, Persson and Rieskamp, 2009, Platzer and Bröder, 2013, von Helversen et al., 2010, von Helversen et al., 2013).
Section snippets
Study 1
Study 1 examined if relying on a rule versus relying on similarity leads to different information retrieval from memory, which, in turn, is reflected in different eye movements. Participants had to decide if job candidates applying for a position were suitable, that is, whether they should be invited for an interview or rejected. In a training phase participants learned information about two suitable and two unsuitable job candidates. In a subsequent test phase they were instructed to decide if
Study 2
The purpose of Study 2 was to test if information retrieval from memory also differs between rule-based and similarity-based decision strategies when the strategy is employed spontaneously. To be able to compare explicit and spontaneous strategy use we investigated the eye movements related to memory processes when strategies are spontaneously employed and when explicit instructions are given to use a specific strategy.
Research in categorization, judgment, and decision making based on cognitive
General discussion
When making everyday decisions from memory people can apply abstract rules that process the available information for a decision or they can make a decision according to similar decision situations encountered in the past (Ashby et al., 1998, Erickson et al., 1998, Juslin and Persson, 2002, Nosofsky et al., 1994, Platzer and Bröder, 2013). Although this distinction is intuitively appealing it proves hard to separate on a process level. One reason is that the two processes are conceptually
Conclusion
By observing eye movements while people performed memory-based decisions using a similarity-based or a rule-based strategy, we showed that the two strategies involve different memory processes. Although similarity and rule users had built the same memory representations, they differed in how these representations were accessed when making a decision. Whereas similarity users retrieved information about similar exemplars, rule users did not—providing empirical evidence that the two processes can
Acknowledgments
The authors would like to thank Claudia Dietzel, Jet Hoe Tang, Regina Weilbächer, Stefan Thommen, Nancy John, and Christian Amstad for their help in conducting the experiments and Josef Krems for encouraging this research. The research was supported by Grant D/12/43949 from the German Academic Exchange Service to the first author and Swiss National Science Foundation research Grants 100014_130192 and 100014_146169 to the second and third author.
References (85)
- et al.
Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models
Journal of Memory and Language
(1998) - et al.
The real-time mediation of visual attention by language and world knowledge: Linking anticipatory (and other) eye movements to linguistic processing
Journal of Memory and Language
(2007) - et al.
Category learning and multiple memory systems
Trends in Cognitive Sciences
(2005) - et al.
Taking a new look at looking at nothing
Trends in Cognitive Sciences
(2008) - et al.
Similarity and rules: Distinct? Exhaustive? Empirically distinguishable?
Cognition
(1998) - et al.
Exemplar similarity and rule application
Cognition
(2010) - et al.
When facts go down the rabbit hole: Contrasting features and objecthood as indexes to memory
Cognition
(2008) - et al.
Looking, language, and memory: Bridging research from the visual world and visual search paradigms
Acta Psychologica
(2011) - et al.
Memory indexing of sequential symptom processing in diagnostic reasoning
Cognitive Psychology
(2014) - et al.
Use of current explanations in multicausal abductive reasoning
Cognitive Science
(2001)