Measuring strategic control in artificial grammar learning

Abstract

In response to concerns with existing procedures for measuring strategic control over implicit knowledge in artificial grammar learning (AGL), we introduce a more stringent measurement procedure. After two separate training blocks which each consisted of letter strings derived from a different grammar, participants either judged the grammaticality of novel letter strings with respect to only one of these two grammars (pure-block condition), or had the target grammar varying randomly from trial to trial (novel mixed-block condition) which required a higher degree of conscious flexible control. Random variation in the colour and font of letters was introduced to disguise the nature of the rule and reduce explicit learning. Strategic control was observed both in the pure-block and mixed-block conditions, and even among participants who did not realise the rule was based on letter identity. This indicated detailed strategic control in the absence of explicit learning.

Introduction

During implicit learning, knowledge acquisition occurs largely unintentionally and at least the details of the learned knowledge are not consciously available to the learner. In implicit learning research, it is becoming increasingly common to measure the degree of strategic control that participants have over their acquired knowledge – i.e., to measure the ability to use the knowledge in a flexible, controlled, context dependent manner (Destrebecqz and Cleeremans, 2001, Destrebecqz and Cleeremans, 2003, Fu et al., 2010, Norman et al., 2006, Norman et al., 2007, Wilkinson and Shanks, 2004). In this paper we suggest some limitations in the way that strategic control has been measured in artificial grammar learning (AGL) which is one of the most studied implicit learning paradigms. We then present an experiment which demonstrates how measurement of strategic control can be improved, and which assesses whether strategic control over knowledge is possible without full explicit knowledge of what has been learned.

Following theoretical work on consciousness by Baars, 1988, Baars, 2002, and the process dissociation procedure of Jacoby and colleagues (Jacoby, 1991, Jacoby, 1994, Jacoby et al., 1993), strategic control is often taken as a hallmark of conscious or explicit knowledge rather than of unconscious knowledge whose influences are much more automatic. Indeed strategic control has been used methodologically as a broad criterion to decide whether acquired knowledge should be regarded as conscious and explicitly learned, or as unconscious and implicitly learned. For example some studies have argued that the learning of sequences of visual target positions in the serial reaction time task (SRT) – as measured by reaction times of rapid key presses to indicate consecutive target positions – is implicit since participants cannot deliberately withhold the influence of their learned knowledge in a generation task using “exclusion” instructions (Destrebecqz and Cleeremans, 2001, Destrebecqz and Cleeremans, 2003, Goschke, 1998). By contrast other studies have argued that such learning is explicit since they found the influence of learned knowledge can be deliberately withheld (Wilkinson & Shanks, 2004).

Moving beyond a simple dichotomisation between explicit and implicit learning, measurement of strategic control has been included alongside other subjective and objective measures of learning to assess exactly which aspects of acquired knowledge participants are conscious of. For example, Dienes, Altmann, Kwan, and Goode (1995) have explored strategic control in artificial grammar learning (AGL). In a typical AGL experiment (Reber, 1967), participants are first presented with a series of letter strings in which the identity and order of letters is governed by a complex rule set referred to as a finite state grammar. Strings that conform to the rule set are “grammatical” and strings that violate it are “ungrammatical”. However, participants are not informed of the existence of a grammar during training. The structure of the grammar can be conceived of as a network of nodes between which only some transitions are permitted, with each legal transition corresponding to a letter (see Fig. 1). In a subsequent test phase, participants are informed of the existence of a grammar and asked to classify whether each of a series of novel letter strings follows this grammar or violates it.

The innovation introduced by Dienes et al. (1995) was to passively expose all participants in an AGL experiment to two sets of letter strings obeying two different finite state grammars (A or B). They then tested participants’ ability to classify whether a novel set of letter strings followed specifically one of these grammars. Within any block of trials the target grammar for string classification was either always A or always B. Participants could classify novels strings with above-chance accuracy in all experiments, regardless of whether they were told to classify according to rule set A or rule set B. These experiments therefore showed strategic control over rule knowledge. From the perspective that strategic control should be seen as a hallmark of conscious knowledge, one interpretation is that learning was explicit. This was also supported by an overall significant relationship between classification accuracy and confidence ratings in two out of three experiments. However, strategic control also occurred on the subset of trials where participants rated that they guessed their classification response. It has therefore been argued by Dienes et al. (1995) that participants might be able to strategically control which grammar they are applying even when they are not conscious of the details of the rules of that grammar. Later, Dienes and Scott (2005) distinguished between judgement knowledge, which refers to knowledge of whether particular items are grammatical, and structural knowledge, which refers to knowledge of the rules of the grammar. The findings by Dienes et al. (1995) could be considered as an example of strategic control in the absence of conscious structural knowledge. A similar finding was reported in a recent study by Wan, Dienes, and Fu (2008) who found strategic control over knowledge of two artificial grammars to which people had been exposed, even on trials where participants rated their grammar classification response to be based on intuition, familiarity, or even random choice rather than rule awareness.

However, the procedure used to measure strategic control in these experiments is not without limitations. Participants in the studies of Dienes et al. and Wan et al. successfully attempted, during any given block of test trials, to classify novel letter strings according to only one of two learned grammars. However, as suggested by Norman et al. (2006), successful performance on this type of AGL task may reflect an ability to voluntarily control whether or not acquired knowledge of a rule is generally activated and allowed to influence task performance, rather than reflect moment by moment control over acquired knowledge. Switching the target grammar from trial to trial would therefore be a more demanding test of strategic control than instructing participants to activate one mental set rather than another at the start of a block of trials (i.e., instructing to use grammar A vs. use grammar B). If a dissociation between strategic control and metacognitive awareness could be shown even when participants are required to switch between the grammars from trial to trial this would provide stronger evidence for strategic control in AGL.

Note that the concerns related to Dienes et al.’s pure-block procedure also apply to a two-grammar AGL study by Higham, Vokey, and Pritchard (2000) where participants showed some ability to classify whether novel strings were from grammar A or from grammar B or from neither. Although participants had to maintain two active grammar representations at the same time, and judge which was closest to the current letter string, they did not have to switch mental sets and selectively control the application of each grammar. (It might even be argued that applying both grammars on any one trial is less demanding than maintaining one grammar while inhibiting another over the course of a block as in Dienes et al. or Wan et al.).

Similar criticism of how strategic control is measured has already been addressed in the serial reaction time task (SRT), which is another widely used implicit learning paradigm (Nissen & Bullemer, 1987). In the SRT task participants observe a simple target such as a circle that appears in rapid succession in different screen locations. The locations follow a repeating sequence. Learning of the sequence is measured partly by the reaction times of rapid key presses to indicate consecutive target positions, and partly by participants’ ability to freely generate key press sequences that follow the learned rule. In the context of the SRT task, the ability to comply with instructions to avoid generating legitimate sequences has been proposed as an example of strategic control (Destrebecqz and Cleeremans, 2001, Goschke, 1998). However, it has been argued that participants might be able to strategically suppress the influence of learned sequence knowledge by adopting response strategies such as perseveration of key presses (Wilkinson & Shanks, 2004), or via a global inhibition of sequence knowledge that does not require any trial-by-trial control over the knowledge (Dienes and Scott, 2005, Norman et al., 2006). Norman et al. (2007) therefore developed a more demanding measure of strategic control that was immune to these criticisms. They showed that participants in a modified SRT experiment could successfully use their “intuitive” sequence knowledge to predict the next target move in a flexible manner according to stimulus–response mappings that varied randomly from trial to trial.

We now report an experiment that tests whether trial-by-trial strategic control is also possible in the context of an AGL task. All participants were first exposed to two sets of stimuli consisting of letter strings structured according to two different finite state grammars taken from Dienes et al. (1995) and Reber (1969) (see Fig. 1). Grammar knowledge was later tested by asking participants on each test trial to pick which of three simultaneously-presented letter strings conformed to one of the target grammars. A between-subjects design was used. For participants in a pure-block condition the target grammar in the test phase was always grammar A or always grammar B, even though all participants were initially exposed to both grammars. For participants in a mixed-block condition, the target grammar varied randomly from trial to trial during the test phase.

We also wanted to determine whether any ability to select target grammars under the more strategically demanding mixed-block procedure would be accompanied by sufficient metaknowledge to generate predictive confidence ratings. To this end we asked participants to give confidence ratings of their grammar judgement at the end of each trial.

In order to reduce the likelihood of explicit learning, and to facilitate our ability to identify participants who did not develop explicit knowledge, we made an important modification to the appearance of the stimuli used in our study. One aspect of the standard AGL paradigm that may promote explicit learning at the cost of more implicit learning is that the only stimulus dimension that varies between individual letter strings is the identity and order of the letters. This makes it easy for participants to develop a general understanding of the nature of the rule, i.e., that it indeed has to do with the identity and order of letters. Participants will readily suspect this during the training phase even if they are not informed of the existence of rules. The likelihood that participants will engage in conscious hypothesis-testing is therefore increased, perhaps resulting in the development of explicit knowledge of at least fragments of the rules (see e.g., Wilkinson & Shanks, 2004). Using a technique introduced by Norman et al. (2007), we therefore modified the AGL task so that it was less apparent that the rule had to do with letter sequence. Norman et al. showed that trial by trial flexibility in the use of rule knowledge in the SRT task could still be expressed when the nature of the rule that governed target position was disguised by random variation in the shapes and colours of targets, even for participants who were now unaware that the rule was based on sequences of locations. Following Norman et al. (2007), we attempted to disguise the fact that our artificial grammars were based on letter sequences by randomly varying the colour and font in which the letters were presented. Even if some participants still realised that the rule was probably based on letter identity, the added perceptual complexity of the displays might still make it more difficult for them to develop explicit knowledge of the rule sets.