Relations between timing, position, and grouping in short-term memory

Abstract

This article is concerned with how information about time and position in a sequence is represented in short-term memory and expressed in the dynamics of serial recall. Temporal-distinctiveness theories of memory predict that isolating a list item in time will improve recall accuracy for that item. Although the majority of research in short-term memory has failed to demonstrate a temporal isolation effect (TIE), there are occasions on which a TIE is observed. The disparity in results has been explained by assuming that participants can adaptively weight temporal and nontemporal information at retrieval, with differences between experiments promoting or discouraging reliance on time as a source of episodic information. A particular focus of the present study is the finding that the TIE is substantially observed in standard serial recall only when participants are instructed to group the list into minisequences. The findings of two experiments using instructed grouping replicated this effect but showed that it is attributable to a longer gap at the group boundary enhancing the positive effect of grouping on recall accuracy. These results show that the hierarchical representations usually associated with temporal grouping are also elicited by instructed grouping but that an additional and nonspecific benefit to recall obtains from lengthening the pause between groups. An additional role for time is identified in the timing of responses: The dynamics of input sequences tend to be mirrored in output sequences for ungrouped lists, whereas the primacy pattern in grouped lists is for a longer duration to speed access to the following group when that duration occurs at an instructed group boundary.

A major principle of memory promoted by a number of theorists is distinctiveness: Items that are more easily discriminated from other items in memory are more accurately recalled (e.g., Brown, Neath, & Chater 2007; Hunt, 2003; Murdock, 1960; Roediger & Crowder, 1976). A recent general model of distinctiveness in memory is the SIMPLE model (Scale Invariant Memory, Perception, and Learning) of Brown et al. (2007). A core principle of SIMPLE is temporal distinctiveness: Events are arranged along a temporal dimension, and distinctiveness along this temporal dimension at least partly determines the confusability of memories (see also, e.g., Bjork & Whitten, 1974; Brown, Preece, & Hulme 2000; Glenberg & Swanson, 1986). Under the assumption that the temporal dimension is logarithmically compressed, such that recent items are more distinct, the model can explain an impressive number of memory phenomena, including recency effects in free recall; forgetting over time and its sensitivity to proactive interference; release from proactive interference; positional confusions in ungrouped and grouped lists in serial recall, over both short and long delays; the phonological similarity effect in serial recall and its reversal with delay; and isolation effects (Brown et al., 2007). Although it specifies a single, simple principle extending over all time scales, with some further assumptions about processes of rehearsal (which are borne out by the data: Brown, Della Salla, Foster, & Vousden 2007), the model also accounts for findings usually interpreted as evidence for dual-store models of memory, particularly dissociations between primacy and recency across amnesic patients and controls (Neath & Brown, 2006).

SIMPLE has motivated a great deal of recent research in the short-term memory literature because it makes the clear prediction that items presented close together in time should be more easily confused than items separated by a large gap. By implication, items that are surrounded by larger temporal gaps will be more temporally distinct, and therefore more accurately recalled. Such an effect is also predicted by the OSCAR (OSCillator-based Associative Recall) model of Brown et al. (2000)—in which time-varying oscillators code the position of items—and also follows from a time-based interpretation of the phonological loop model of Burgess and Hitch (1999). In contrast, event-based models assume that items are not coded temporally, and that ordinal representation is based solely on the presentation of items (Farrell & Lewandowsky, 2002; Lewandowsky & Farrell, 2008). Although some earlier evidence had suggested that temporal isolation does enhance recall (e.g., Neath & Crowder, 1990, 1996), Lewandowsky, Wright, and Brown (2007) argued that those experiments were not diagnostic because the temporal separation of items was very predictable, given the use of, for example, temporal intervals that monotonically increased or decreased across output positions. Lewandowsky and Brown (2005) found that when this predictability was controlled by pseudorandomly mixing temporal intervals, there was no effect on accuracy of the duration preceding an item, but longer durations following an item were indeed associated with more accurate recall. They suggested that the effect of the duration following an item was due to a longer interval allowing increased rehearsal and consolidation of the immediately preceding item and showed that requiring participants to carry out articulatory suppression, which presumably blocks these processes, resulted in no effect of either preceding or following duration.

Since the publication of their initial findings, Lewandowsky and Brown’s (2005) finding of a lack of a temporal isolation effect (TIE) on serial recall accuracy has been replicated in a number of tasks and modalities. Temporally isolating an item does not enhance serial recall of that item, regardless of overall presentation rate (Nimmo & Lewandowsky, 2005), presentation modality (Nimmo & Lewandowsky, 2006), serial position (Lewandowsky, Brown, Wright, & Nimmo 2006), and regardless of whether serial recall, probed recall, or serial recognition is used to probe memory (Farrell & McLaughlin, 2007; Lewandowsky et al., 2006). No significant TIE is observed for serial recall of visuospatial materials (Parmentier, King, & Dennis 2006), and surprisingly, a TIE fails to be observed when participants are explicitly required to attend to the timing of items by postcueing for recall or recognition of the order of items or their timing (Farrell, 2008; Farrell & McLaughlin, 2007). Nonetheless, a TIE is observed in running memory span (Geiger & Lewandowsky, 2008) and in free recall (Brown, Morin, & Lewandowsky, 2006), and when participants are allowed to reconstruct the order of items in an input sequence in any output order (Lewandowsky, Nimmo, & Brown, 2008). In the latter case, the TIE is modulated by the output task even in cases where the nature of the output task is postcued. More recently, Morin, Brown, and Lewandowsky (2010) suggested that temporal isolation does have effects on serial recall, but that the effect is small and that previous studies have not had sufficient power to reliably detect a TIE. On the basis of these findings, Lewandowsky et al. (2008) argued that SIMPLE must include both a temporal dimension (producing a TIE) and a positional dimension (which does not produce a TIE), and that participants can selectively weight these dimensions at recall on the basis of the recall task. In the case of serial recall, the implication is that the temporal dimension is only weakly weighted, given the numerically small and nonsignificant isolation effects typically observed.

The possibility that both positional and temporal dimensions might contribute to the recall of items was used by Farrell (2008) to explain one puzzling violation of the near-universal failure to observe a TIE in serial recall. In an experiment in which participants were precued to recall either the order or the timing of items, Farrell (2008) found a TIE when participants were instructed to think of the lists as being grouped (for example, to think of the list “7 5 8 4 1 2 9 3” as composed of two sub-sequences: “7 5 8 4” and “1 2 9 3”). Instructing participants to group lists produced effects similar to those observed under temporal grouping, including nonmonotonicities in accuracy serial position function and increased latencies at the group boundary in the latency serial position function. The universal account of temporal grouping effects in contemporary serial recall models is that groups are treated as “mini-lists,” such that the position of items is represented along two dimensions: one representing the position of the item in its group, and a second, coarser dimension representing the position of the item or group in the list as a whole (Brown et al., 2000; Burgess & Hitch, 1999; Henson, 1998; Lewandowsky & Farrell, 2008). To account for grouping effects in the SIMPLE model, Brown et al. (2007) assumed that the position of items within a group is coded along a positional dimension and that a second temporal dimension codes for a coarser representation of position. This not only explains the effects of grouping on accuracy and confusions between lists, but also explains why confusions of items between groups are based on their shared within-group position, not on time of presentation in the group (Ng & Maybery, 2005). Farrell (2008) explained the observation of a TIE in grouped but not ungrouped lists with reference to these different modes of positional representation in SIMPLE. Farrell (2008) suggested that asking participants to group lists requires them to recruit the temporal dimension to represent overall position of items in the list; otherwise, items sharing the same within-group position would be indistinguishable. As long as attention is being paid to this temporal dimension, a TIE is produced.

This framework actually points to several ways in which timing could be involved in short-term memory. One possible role of time is its use as a cue to the hierarchical structuring of a list. As mentioned above, participants appear to enforce a hierarchical structure if they are asked to think about the items on a list in a grouped fashion (e.g., Farrell, 2008). A more traditional method for eliciting these grouping effects has been to manipulate the timing of the items, leaving an extended pause at the boundary between groups (e.g., Farrell & Lelièvre, 2009; Henson, 1999b; Hitch, Burgess, Towse, & Culpin 1996; Ryan, 1969a). A reasonable suggestion might be that an extended pause leads to isolation at a group level; that is, items are represented along a dimension of time, and leaving an extended pause in the sequence separates the groups of items adjacent to the pause. However, this explanation cannot explain the efficacy of other types of grouping cues, such as instructed grouping (e.g., Farrell, 2008), pitch (Frankish, 1989), prosody (Frankish, 1995), and location (Anderson & Matessa, 1997; Parmentier & Maybery, 2008), in producing effects similar to those of temporal cues (though not quantitatively as large; see, e.g., Parmentier & Maybery, 2008; Ryan, 1969b). Indeed, the suggestion that participants spontaneously group lists that are presented in a homogeneous fashion (e.g., Farrell & Lelièvre, 2009; Henson, 1998; Madigan, 1980) suggests that these grouping effects are due to top-down processes that are nonetheless sensitive to the arrangement of sequence elements.

A second possibility is that the time of presentation of items is represented in some fashion. In SIMPLE, this is accomplished by positioning items on a logarithmically compressed temporal dimension. A mechanistic analogue of SIMPLE’s temporal dimension is found in the OSCAR model of Brown et al. (2000), in which a bank of oscillators, whose activity changes sinusoidally over time, generate a temporal context that represents the positions of items in the list. Each item is associated with the temporal context prevailing at its presentation, and list items are retrieved in order by “rewinding” and replaying the successive temporal contexts and retrieving their associated items. Both models, along with others (Burgess & Hitch, 1999; Henson, 1998; Lewandowsky & Farrell, 2008), also assume that the order of items is represented hierarchically, with one dimension or set of units representing position within a group, and another representing position in the list as a whole. Where SIMPLE and OSCAR differ is in the necessity of a contribution of the temporal dimension: In SIMPLE, the temporal dimension is attentionally weighted so that it can contribute a lot or a little (Lewandowsky et al., 2008), while in OSCAR the oscillators representing within-group position and position in the entire list are necessarily temporally based.

Finally, time might be represented in short-term memory as part of the information about the dynamics of the sequence. A number of studies have shown that participants can recognize and reproduce the timing of an input sequence (Collier & Logan, 2000; Farrell, 2008; Farrell & McLaughlin, 2007; Watkins, LeCompte, Elliott, & Fish 1992). On the basis of a failure to find a TIE in the face of the ability to remember the timing of short sequences, Farrell (2008) suggested that participants remember the individual durations as a form of information that is associated with, and can be retrieved by, an ordering mechanism such as a positional marker. Intriguingly, Farrell (2008) found that serial recall of nontemporal information was also affected by the dynamics of the sequence: A longer gap between items on the list produced a longer pause in participants’ responses, even when they were asked simply to recall the list items. Farrell (2008) suggested that recall might consist of the construction of “output plans” incorporating both order and timing information, as suggested by Tubau, Hommel, and Lopez-Moliner (2007) in the case of sequence learning, consistent with the more general notion that recall consists of the reenactment of list events (e.g., Kent & Lamberts, 2008).

This article presents two experiments that make some attempt to tease apart these different possible roles of time in short-term memory. In a departure from Farrell’s (2008) examination of the TIE for grouped lists, the experiments here never required participants to remember the timing of items, strengthening the argument that any temporal effects found reflect some core property of the processes or representations underlying short-term memory for sequences. A second key difference here is the examination of effects according to serial position. As we show below, the TIE observed by Farrell (2008) is likely to be a result of participants strategically using the temporal gaps to more effectively implement grouped representations in short-term memory, such that a primary involvement of time in short-term memory is to facilitate the hierarchical structuring of sequences.

Experiment 1

To investigate TIEs, participants were presented with lists of eight digits presented with irregular timings. Halfway through the experiment, participants were instructed to think of the list as two separate groups of four items. Instructing participants to think about lists as separate groups is known to elicit grouping effects (Ryan, 1969b) but does not interfere with the irregular timings used during item presentation, which would be inevitable with temporal grouping. Item accuracies were then examined in relation to the degree of temporal isolation of each item to further investigate the extent to which TIEs may occur.

Method

Design and participants

This experiment had a 2 (instructions: ungrouped vs. grouped) × 8 (serial position) within-subjects design. A within-subjects design was chosen for this and the following experiment to maximize their power, given Morin et al.’s (2010) claim that previous failures to observe TIEs have been Type II errors. Of the 30 participants, 27 were undergraduate students. Participants were between 19 and 29 years of age (M = 22.04, SD = 2.37) and were native or fluent English speakers.

Materials and apparatus

Each list contained eight digits randomly sampled without replacement from the digits 1–9. The sampling in these lists was subject to two constraints: First, there were no pairs of adjacent integers (e.g., 5–6, 9–8); second, no item could appear in the same serial position on successive lists (Farrell, 2008; Henson, 1996).

The stimuli were recorded from a male speaker with a neutral English accent. The pitch was flattened to a fundamental frequency of 113 Hz, typical of an adult male, using the Praat phonetics software (Boersma & Weenink, 2006). This was done because it is known that pitch variations can be used as grouping cues (Frankish, 1995). The stimuli were also normalized to a constant length of 500 ms. Finally, the digits were padded so that they had equal perceptual centers, since the perception of stimulus onset is known to differ from its actual onset (Scott, 1998). Digits were auditorily presented with randomly ordered ISIs of 100, 200, 400, 650, 900, 1,200 and 1,500 ms (Farrell, 2008).

The experiment was run on a PC, and the Psychophysics Toolbox for MATLAB (Brainard, 1997; Pelli, 1997) was used to present stimuli over headphones and to record participants’ responses.

Procedure

Participants were tested individually in the laboratory. Each trial started with the presentation of a fixation cross in the center of the screen, which lasted 1,000 ms. This was followed by a blank screen for 500 ms, after which the digit list commenced. The digits were played to participants over a set of headphones. Following presentation of the list, there was another 500-ms period of blank screen, followed by the cue recall appearing. Participants were then required to type the digit list in the remembered order using the numeric keypad of the keyboard. The recall prompt remained on the screen until eight digits had been entered. Participants were required to produce exactly eight digits and were instructed not to press any other keys. The screen then became blank for 1,000 ms before commencement of the next trial. Once 40 trials had been completed, an instruction screen appeared, informing participants that for the remainder of the experiment they were to “think about the eight numbers on each list as being in two groups of 4.” To ensure that participants understood the instruction, the following example was used: “... if you got the list of numbers ‘1 7 4 5 8 3 2 9’, you would remember ‘1 7 4 5’ as one group, and ‘8 3 2 9’ as another group.”

The experimental task for the remaining 40 trials was otherwise unchanged. The grouping instruction was given in the second half of the experiment for all participants. Although there was potential for a confound due to order effects, counterbalancing would have caused problems because participants who were instructed to group in the first half would have been likely to continue to do so in the second half.

Three practice trials were presented at the start of the experiment to familiarize participants with the procedure and to check that the volume was at a suitable level. There was a self-paced break every 10 trials; participants were instructed by an on-screen message to press the space bar to continue. Upon pressing the space bar, a blank screen would be presented for 2,000 ms before the next trial commenced. The whole experiment lasted approximately 30 min.

Results and discussion

Accuracy

A 2 (grouping instruction: ungrouped vs. grouped) × 8 (serial position) repeated measures ANOVA revealed a main effect of grouping, F(1, 29) = 14.2, p < .005, with higher accuracy in the grouped condition (M = .813) than in the ungrouped condition (M = .758). There was also a significant effect of serial position, F(7, 203) = 44.2, p < .001. As can be seen from the left panel of Fig. 1, the data show clear primacy and recency effects, with higher accuracy at the start and end serial positions. A significant interaction between grouping and serial position was also found, F(7, 203) = 4.54, p < .001; in the grouped condition, mini serial position curves occurred within each group, illustrated by the scalloped appearance of the grouped accuracy curve in the left panel of Fig. 1.

Fig. 1

Accuracy serial position functions for Experiment 1 (left panel) and Experiment 2 (right panel). Error bars here and throughout are within-subjects standard error bars (Bakeman & McArthur, 1996)

To measure TIEs, a mixed-effects linear regression was fit to the data separately for the ungrouped and grouped conditions.¹ Mean accuracy (for internal serial positions only, since the first and last list positions are, respectively, missing a preceding and a following duration) was predicted from the interval preceding (pre) and following (post); random effects were allowed for the two predictors and the intercept. For the ungrouped condition, the effect was significant for neither pre, β = −.0002, t(1223) < 1, nor post, β = .0036, t(1223) = 1.20, p = .23. For the grouped condition, a significant effect was found for post, β = .0058, t(1226) = 2.30, p = .02, but not for pre, β = .0042, t(1226) = 1.55, p = .12.²

This overall analysis appears to be broadly consistent with those of Farrell (2008) in showing a TIE effect for grouped (but not ungrouped) lists, although Farrell (2008) found a significant effect for following, not preceding, durations for grouped lists. However, further analysis revealed that the overall TIE effect observed here is quite restricted. Following Lewandowsky et al. (2006), the above effects were broken down by serial position, with pre and post converted to binary variables by categorizing durations as “short” (100, 200, 400, and 650 ms) and “long” (900, 1,200, and 1,500 ms). The resulting serial position effects on accuracy are shown in Fig. 2 for intervals preceding and following items at Serial Positions (SPs) 2–7. The one obvious pattern in the two figures is that an advantageous effect of a longer duration is obtained only for the duration at the suggested group boundary—that is, the duration preceding SP 5 (left panel) and following SP 4 (right panel). Separate repeated measures ANOVAs for the two plots in Fig. 2 did not reveal the expected three-way interaction between grouping condition, duration, and serial position [preceding, F < 1; following, F(5, 145) = 1.96, p = .088]. This appears to be attributable to a similar pattern in the ungrouped condition, and the interaction between serial position and duration without respect to grouping condition is significant for both plots [preceding, F(5, 145) = 6.37, p < .001; following, F(5, 145) = 3.42, p < .01].

Fig. 2

Recall accuracy for Experiment 1 as a function of preceding pause duration (left panel) or following pause duration (right panel), for ungrouped and grouped lists

One explanation for the pattern in Fig. 2 is that having a larger pause between the groups allows a more effective implementation of the grouping strategy, with some suggestion that a grouping strategy was spontaneously employed for the ungrouped lists, using a 4–4 pattern (see, e.g., Farrell & Lelièvre, 2009). If this is the case, the benchmark effects of grouping should be more pronounced for the entire list if the duration at the group boundary is longer (see, e.g., Maybery, Parmentier, & Jones 2002). Figure 3 plots the entire accuracy serial position function as in Fig. 1, but with the duration intervening between SPs 4 and 5 (i.e., the intergroup pause) as a factor. The results show that having a long duration between SPs 4 and 5 increases performance across the majority of the list, but only for grouped lists. This is reflected both in a significant two-way interaction between time and grouping, F(1, 29) = 6.31, p = .018, with a longer intergroup pause increasing accuracy more for grouped lists (short: .78 vs. long: .85) than for ungrouped lists (short: .75 vs. long: .77), and a three-way interaction from a repeated measures ANOVA, F(7, 203) = 2.47, p = .019, with the recall advantage being mostly confined to the internal serial positions.

Fig. 3

Serial position functions for Experiments 1 (left panel) and 2 (right panel), by grouping condition and the duration between Items 4 and 5

Latencies

To examine the matching of interresponse pauses at output with the corresponding items at input, input durations were divided into discrete categories (short vs. long) as above. The resulting serial position functions for the crossing of input duration with grouping instruction are shown in the top row of Fig. 4; note that serial position here refers to the position of the duration within the ordered set of interitem durations, and is equivalent to the serial position of the preceding item. The latencies were analyzed using a 2 (duration: short vs. long) × 2 (grouping instruction: ungrouped vs. grouped) × 7 (serial position) repeated measures ANOVA. As in the figure, durations were only included in the analysis if both the preceding and following items were recalled correctly, leading to the omission of 1 participant who produced empty cells. Although the trend in the figure is for a longer pause at input to produce a longer pause between responses at output, the ANOVA revealed that the effect of duration was nonsignificant, F(1, 28) = 2.64, p = .115, η² = .09; additionally, duration did not significant interact with grouping instruction, F(1, 28) < 1, or serial position, F(6, 168) < 1, and the three-way interaction was also nonsignificant, F(6, 168) < 1.

Fig. 4

Serial position functions for interitem response durations, plotted by interitem duration in input and grouping instruction

We also analyzed the effect of duration of the intergroup pause on the latency serial position function, complementing the accuracy analysis in Fig. 3; the results are shown in Fig. 5. For this analysis, latencies were included for correct responses (i.e., if the response that terminated that duration was correct). A repeated measures ANOVA revealed a marginally significant three-way ANOVA, F(7, 203) = 1.89, p = .073, η² = .06. Given that the effect was close to significance, and for ease of comparison with results from later experiments, the three-way effect was broken down by running separate two-way (Intergroup Duration × Serial Position) analyses for the ungrouped and grouped conditions. For the ungrouped condition (top left panel of Fig. 5), only the effect of serial position was significant, F(7, 203) = 54.28, p < .001; the effect of duration, F(1, 29) < 1, and its interaction with serial position, F(7, 203) < 1, were nonsignificant. For the grouped condition (top right panel), latencies were significantly longer for a smaller intergroup pause, F(1, 29) = 32.74, p < .001. Along with the main effect of serial position, F(7, 203) = 33.02, p < .001, duration also interacted with serial position F(7, 203) = 3.05, p = .005, although inspection of Fig. 5 doesn’t suggest any systematic pattern, particularly with relation to similar results from later experiments.

Fig. 5

Serial position functions for recall latencies, plotted by grouping instruction and intergroup pause duration. Top row: Experiment 1. Bottom row: Experiment 2

Summary

The results of Experiment 1 provided further evidence that time does have an effect on accuracy for grouped lists, as suggested by Farrell (2008). However, the restriction of this effect to the items immediately surrounding the group boundary, and the extension of the beneficial effect of a longer intergroup duration to all list items, suggests that participants are using the additional time to more effectively employ a grouping structure. The results are partially consistent with a finding reported by Lewandowsky et al. (2006), who, in one experiment, observed a TIE specifically at the fourth serial position in seven-item lists. They suggested that this position-specific TIE might reflect subjective grouping: Some participants used the long pause to subjectively group the list. Lewandowsky and colleagues asked independent observers to classify participants as “groupers” (those given to subjective grouping) and “nongroupers” on the basis of their response latencies. Consistent with their expectations, Lewandowsky et al. (2006) found that the temporal isolation effect was observed only for groupers, and only for items preceding the presumed group boundary. Our results show a TIE for both items adjoining the group boundary, as would be expected from the finding that grouping enhances recall most for the first and last items within each group (e.g., Farrell & Lelièvre, 2009; Henson, 1999a; Ng & Maybery, 2002), and furthermore demonstrate that this beneficial effect extends to other list items. The results appear to rule out other potential explanations for the TIE, such as consolidation and rehearsal, because the beneficial effect should act retroactively in both cases (i.e., the pause cannot allow more time for rehearsal or consolidation of items if those items are yet to be presented).

The latency data do not fundamentally add to this story, as there were no strong and systematic effects. Nonetheless, we highlight two effects that are relevant for the following experiments: One is the effect of input timing on output timing, which approached significance. Consistent with the results of Farrell (2008), the pattern of this trend was for longer durations in the input to produce longer durations in the output. A second pattern that partially worked against this was for a longer duration at the group boundary to speed recall for the entire list. When considered in relation to the accuracy results, a reasonable suggestion is that the longer pause makes grouped lists more accessible, by increasing accuracy and speeding recall for grouped lists. We defer further theoretical interpretation of these results until after presentation of a second experiment.

Experiment 2

Before elaborating on the conclusions, we first address a remaining issue via a second experiment. Several of the effects observed here, including the mirroring of input and output times that approached significance, were not as strong as those observed in Farrell’s (2008) study. One likely explanation is that the larger list length here diluted the effect by increasing the number of temporal durations that needed to be discriminated. To potentially increase the power of the key analyses, a second experiment was run involving only two clearly distinguishable interitem durations.

Method

Participants

Thirty participants took part in Experiment 2, although data from 1 participant were excluded from the analyses because that participant’s mean accuracy score was over two standard deviations below the overall mean. All participants were native or fluent English speakers between 18 and 35 years of age and were remunerated £7 cash for their time.

Materials and procedure

The materials and procedure were the same as in Experiment 1, with a few adjustments. Firstly, ISIs were now either short (100 ms) or long (1,000 ms). Each list contained four short ISIs and three long ISIs in a random order. Also, participants were now given two practice trials before the experiment started, and there were 60 trials in each grouping condition, to give 120 trials overall. As a result, Experiment 2 ran for around 40 min.

Results and discussion

Accuracy

Overall serial position functions are plotted in the right panel of Fig. 1. A 2 (grouping: ungrouped vs. grouped) × 8 (serial position) repeated measures ANOVA revealed a main effect of serial position, F(7, 196) = 45.88, p < .001. Though grouped lists appeared to be recalled more accurately than ungrouped lists, the main effect of grouping did not reach significance, F(1, 28) = 2.88, p = .101. A significant interaction was also found, F(7, 196) = 7.67, p < .001; in the grouped condition, mini serial position curves are apparent, with primacy and recency within each group.

To show the impact of temporal isolation on output accuracy, Fig. 6 plots mean accuracy for SPs 2–7 for each combination of interval duration (short or long) and grouping condition (ungrouped or grouped lists). The left panel plots performance according to the durations preceding item presentation, and the right panel plots accuracy according to the duration following the item. Two separate repeated measures ANOVAs were run for intervals preceding and following items at SPs 2–7 with Time (short or long), Grouping Condition (ungrouped vs. grouped), and SP (2–7) as factors.

Fig. 6

Recall accuracy for Experiment 2 as a function of preceding pause duration (left panel) or following pause duration (right panel), for ungrouped and grouped lists

For preceding intervals (left panel of Fig. 6), significant main effects were observed for serial position, F(5, 140) = 33.29, p < .001, and duration, F(1, 28) = 6.44, p = .017, with the effect of grouping being nonsignificant, F(1, 28) = 2.85, p = .101. The effect of timing was a slightly higher accuracy for items preceded by a long rather than a short interval (short, M = .77; long, M = .78). The interaction between time and grouping condition was marginally significant, F(1, 28) = 3.71, p = .067, with an indication that a long time interval preceding item presentation improved recall accuracy in the grouped but not in the ungrouped condition. However, this and other significant interactions [between grouping and serial position, F(5, 140) = 8.07, p < .001, and time and serial position, F(5, 140) = 5.28, p < .001], appeared to be driven by a significant three-way interaction, F(5, 140) = 4.03, p = .002, with the only apparent advantage from a preceding long interval applying at SP 5, and only for grouped lists.

For intervals following items (right panel of Fig. 6), main effects were observed for serial position, F(5, 140) = 35.44, p < .001, and time, F(1, 28) = 5.61, p = .035, but not for grouping, F(1, 28) = 2.48, p = .127. In addition, a number of interactions were significant [grouping and serial position, F(5, 140) = 9.30, p < .001, and time and serial position, F(5, 140) = 5.96, p < .001], including the three-way interaction, F(5, 140) = 1.22, p = .042. This is a pattern similar to that found for preceding durations, with the advantage of a following long interval being observed only in grouped lists, and only at a single serial position, this time SP 4 (the item preceding the group boundary).

Considering the data in Fig. 6 overall, any advantage gained from a long temporal gap applies only to the items preceding and following that gap when it is placed at the group boundary (i.e., between SPs 4 and 5) in the grouped condition. The right panel of Fig. 3 shows performance on the entire list as a function of grouping instruction at the duration of the pause between groups, and generally conforms to the pattern seen in Experiment 1 (left panel): A long pause at the group boundary enhances performance at the majority of serial positions under grouping instructions. This effect is confirmed by a significant two-way interaction between grouping instruction and duration between SPs 4 and 5, F(7, 196) = 4.37, p < .001, η² = .13, which is qualified by a three-way interaction between grouping instruction, duration at boundary, and serial position, F(7, 196) = 4.37, p < .001, η² = .13; Figure 3 suggests that this is due to a larger beneficial effect of duration for intermediate serial positions. Curiously, the serial position function for the grouped condition with a short pause between SPs 4 and 5 gives some indication of grouping (i.e., a scalloped serial position function), but does not show the enhanced recall often associated with grouping (this is suggested to a lesser extent in the data of Experiment 1 in the left panel).

Latencies

The bottom row of Fig. 4 shows the interitem pauses at output as a function of the input duration and duration position. The latencies were analyzed using a 2 (duration: short vs. long) × 2 (grouping instruction: ungrouped vs. grouped) × 6 (serial position) repeated measures ANOVA, with SP 4 (the group boundary) included. As in the figure, durations were only included in the analysis if both the preceding and following items were recalled correctly. The ANOVA revealed that the effect of duration was clearly significant, F(1, 28) = 14.24, p < .001, η² = .34, with the mean output duration being longer for long (M = 515.38 ms) than for short (M = 471.34 ms) input durations. This effect interacted marginally with grouping instruction, F(1, 28) = 3.53, p = .071, η² = .11, and significantly with serial position, F(6, 168) = 7.01, p < .001, η² = .20; these effects were qualified by a significant three-way interaction, F(6, 168) = 4.34, p < .001, η² = .13. The figure suggests a complicated pattern that is consistent with an overall effect of duration on recall latencies being combined with a specific effect of grouping. Specifically, it appears that duration had a simple effect in the ungrouped condition (longer durations at input leading to longer durations at output); a similar effect prevailed in the grouped condition, with the exception of SP 4 (the group boundary), where a shorter input duration led to a longer output duration. This was confirmed by rerunning the ANOVA with the fourth position excluded. The three-way interaction was no longer significant, F(5, 140) = 1.27, p = .279, η² = .04, as was also the case for the interaction between grouping instruction and duration, F(1, 28) < 1. The main effect of duration remained significant, F(1, 28) = 32.58, p < .001, η² = .54, as was the interaction with serial position, F(5, 140) = 5.35, p < .001, η² = .16, with the effect of grouping being larger at medial serial positions. Examining only those latencies corresponding to the group boundary with a 2 (grouping instruction) × 2 (duration) ANOVA revealed a significant two-way interaction, F(1, 28) = 9.22, p = .005, η² = .25, whereby a longer duration marginally increased latencies in the ungrouped condition, t(28) = 1.81, p = .080, and significantly reduced latencies in the grouped condition, t(28) = 2.17, p = .038. The results suggest that although there was a general tendency for a longer input duration to produce a longer output duration, a longer pause at the group boundary facilitates the employment of a grouping strategy, speeding access to items adjacent to the group boundary.

This was examined further by analyzing the effect of duration of the intergroup pause on the latency serial position function, complementing the accuracy analysis in Fig. 3; results are shown in the bottom row of Fig. 5. A repeated measures ANOVA revealed a significant three-way interaction, F(7, 196) = 3.73, p < .001, η² = .12, which was broken down by running separate two-way (Intergroup Duration × Serial Position) analyses for the ungrouped and grouped conditions. For the ungrouped condition, the effect of duration failed to reach significance, F(1, 28) = 3.08, p = .090. There was a main effect of serial position, F(7, 196) = 43.68, p < .001, and a marginally significant interaction between grouping and duration, F(7, 196) = 1.98, p = .059. Examination of the bottom left panel of Fig. 5 suggests no principled interpretation of the interaction, although the increased latency at the group boundary is reminiscent of the effects of grouping on serial recall discussed earlier, suggesting that a longer pause encouraged participants to spontaneously group in the ungrouped condition. Under instructed grouping conditions, a longer pause at the group boundary led to quicker recall times, F(1, 27) = 44.40, p < .001. This was supplemented by a significant effect of serial position, F(7, 189) = 27.12, p < .001, and an interaction between duration and serial position F(7, 189) = 4.94, p < .001. The interaction was again not interpretable in any systematic fashion (bottom right panel), but there was a suggestion of a larger effect of duration at the beginning of groups (SPs 1 and 5).

Summary

The results of Experiment 2 broadly replicated those of Experiment 1 in showing that longer interitem durations can give rise to more accurate recall for grouped lists. As for Experiment 1, however, more fine-grained analyses suggest that this effect arises merely from a longer pause at the group boundary giving rise to a more pronounced grouping effect (see also Lewandowsky et al., 2006). In addition, input timing was reflected in the timing of responses (a trend that was not significant in Experiment 1), with an increase in response latency with increasing duration between items at input. This relationship broke down at the group boundary (again, in a more obvious fashion than for Experiment 1), where a longer duration produced quicker, rather than slower, output times.

General discussion

The results from the two experiments point to several associations between the temporal characteristics of a sequence and its representation in memory. First, although a temporal isolation effect was observed, particularly for sequences presented under instructed grouping conditions, this effect was attributable to the benefit obtained from having a longer pause at the instructed group boundary. Second, participants had a tendency to leave a longer duration between items in their recall when a longer pause was present in the input sequence (with one exception, discussed below), suggesting that some information about the timing of the sequence was encoded, retained, and expressed at recall.

Grouping, timing, and the temporal isolation effect

One hypothesis, outlined in the introduction, is that a temporal isolation effect will be observed in serial recall of temporally grouped but not of ungrouped lists. The SIMPLE model (Brown et al., 2007) suggests the possibility that the multidimensional nature of the representations underlying grouped lists (e.g., Brown et al., 2000; Burgess & Hitch, 1999; Henson, 1998) necessitates the recruitment of a temporal dimension that will produce a TIE in grouped lists, whereas ungrouped lists will either recruit only an ordinal dimension insensitive to the effects of time (Farrell, 2008) or weight the temporal dimension only very weakly (Morin et al., 2010). On the surface, the results of the first two experiments were consistent with that assumption. However, a more fine-grained exploration of the data revealed that this effect was restricted to the benefit obtained from placing a pause at the boundary between groups, with this advantage applying to all items in the list.

These results provide further evidence against pure temporal distinctiveness accounts of serial recall, in particular the SIMPLE model of Brown et al. (2007). SIMPLE predicts that as long as some attention is paid to the temporal dimension, temporally separating a list item from its neighbors should enhance recall of an item, as it will be relatively more temporally distinct. A large number of experiments (Lewandowsky & Brown, 2005; Lewandowsky et al., 2006; Lewandowsky et al., 2007; Nimmo & Lewandowsky, 2005, 2006; Parmentier et al., 2006), including Experiments 1 and 2 reported here, have failed to find a TIE in serial recall of ungrouped lists when durations are unpredictable. This suggests that people give little if any weighting to the temporal dimension in serial recall, and that memory for order is mostly or wholly dependent on a nontemporal ordinal code. One caveat is a meta-analysis conducted by Morin et al. (2010), who showed that most serial recall studies conducted in the past several years have numerically produced a TIE even though the effect in individual experiments was not significant. Morin et al. (2010) argued that the TIE in serial recall is consistent but small, requiring a fair amount of power to be detected. Numerically, the average size of this effect is observed to be extremely small in the analysis of Morin et al., with a predicted increase in recall accuracy of 1% per second of isolation, suggesting a minimal role for time. Unsurprisingly, the same group of authors (Lewandowsky et al., 2007) have suggested that there is unlikely to be compelling evidence of an important role for time underlying the collected data.

In the experiments reported here, the effects, on average, were small for ungrouped lists: Experiment 1 revealed a maximum enhancement of 0.36% per second of isolation, while Experiment 2 revealed a maximum effect of around 1.3% on recall accuracy from the 900-ms difference between the short and long input durations. In the case of the grouped lists examined here, the effects of temporal isolation were clearly significant but not generic, applying only to the pause at the group boundary: The longer the pause, the greater the enhancement of list items. This grouped-list enhancement may also explain the numerically small but apparently consistent effect for standard (ungrouped) serial recall argued for by Morin et al. (2010), as follows: There is some evidence that participants spontaneously group ungrouped lists (Madigan, 1980), such that ungrouped lists are averages across unknown grouping patterns employed by participants. By examining estimates of mixing of ungrouped and grouped strategies in a computational model, Farrell and Lelièvre (2009) found a wide variety of estimates of the extent of spontaneous grouping in serial recall; and Lewandowsky et al. (2006) observed that differences in spontaneous grouping could lead to the presence or absence of a TIE. As long as there is at least a moderate chance that an ungrouped list will be grouped (see Fig. 12 of Farrell & Lelièvre, 2009), and given that a longer-than-average duration will fall at the group boundary half the time (assuming a roughly symmetric distribution of interitem pauses), it is not surprising that a small but consistent TIE would emerge across experiments.

One remaining question is why a TIE is very clearly observed in other memory tasks, such as free recall (Brown et al., 2006), recognition (Morin et al., 2010), and running memory span (Geiger & Lewandowsky, 2008). One explanation, offered in the preceding reports, is that the temporal dimension that produces a TIE is used in most memory tasks to distinguish between episodes in memory, but that in the serial recall task a positional dimension that does not produce a TIE is used to order items, in preference to the temporal dimension. However, an alternative explanation—following from the results of Experiments 1 and 2—is that participants spontaneously cluster long sequences of information (such as those presented in typical free recall, recognition, and running memory span studies) into smaller groups based on temporal proximity. Accordingly, a longer temporal gap between two list items may both encourage grouping, by forming a temporal discontinuity that suggests a group boundary, and additionally facilitate temporal grouping where a predetermined group boundary and the longer gap coincide. This suggestion receives some support from Farrell (2009), who simulated performance on free recall using a temporal grouping model of serial recall (similar to, e.g., Brown et al., 2000; Burgess & Hitch, 1999; Henson, 1998; Lewandowsky & Farrell, 2008) and showed that assuming spontaneous grouping in free recall was sufficient to produce a TIE.

A second question is, what happens during a longer intergroup pause to produce a benefit for grouped lists? Previous research has shown that under instructed grouping conditions, an intergroup pause of as little as 80 ms is sufficient to produce an advantage over ungrouped lists (Frankish, 1985, 1989; Ryan, 1969a). However, the effect of the length of the pause depends on presentation modality: Although lengthening the intergroup pause produces an increase in accuracy under visual presentation (Frankish, 1989; Maybery et al., 2002), no such benefit is obtained with auditory presentation (Frankish, 1989; Ryan, 1969b). On the basis of this observation, along with the finding that grouping auditorily presented lists by location and pitch produces effects similar to temporal grouping (Frankish, 1989; see also Parmentier & Maybery, 2008), Frankish (1989) argued that grouping effects for auditory lists reflect the use of suprasegmental information by an auditory memory system (akin to the precategorical acoustic store: e.g., Crowder & Morton, 1969) to structure auditory information. Accordingly, extended pauses for auditory lists serve merely as anchor points indicating the end of a group or phrase. However, the results of Experiments 1 and 2 suggest that this view is incomplete, as extended pauses only produced grouping effects when they coincided with the instructed group boundary. Accordingly, any bottom-up information provided by those suprasegmental cues appears to have been modulated by top-down control of the grouping structure instructed by the experimenter. Additionally, extending the intergroup pause has been found to benefit auditorily presented lists in other studies (Maybery et al., 2002).

Given that similar qualitative effects of grouping on serial position functions for accuracy and latencies are found across visual and auditory presentation, and given confusions within and between groups (see, e.g., Farrell & Lelièvre, 2009; Farrell & Lewandowsky, 2004; Henson, 1999b; Maybery et al., 2002; Ryan, 1969a, b), a more concise explanation would be that a single system supports short-term memory for information, irrespective of presentation modality. The clear effects of modality in increasing recency within groups (e.g., Frankish, 1985) suggest that there are some differences due to presentation modality; however, these can be captured as quantitative changes in positional representations (Henson, 1998), although the mechanism by which this occurs has not been fleshed out.

This still leaves unanswered the question of the origin of the beneficial effects of the intergroup pause. Ryan (1969b) suggested that this time was used to recode the items in the preceding group in terms of their within-group position. On the basis of her failure to find an effect of the length of the intergroup pause, Ryan concluded that this process must act quickly, prior to the decay of the preceding items. However, more recent demonstrations of a beneficial effect of lengthening the intergroup pause (e.g., Frankish, 1989; Maybery et al., 2002) call for some revision of Ryan’s (1969b) suggestion. One additional and informative finding is that the extended pause does not appear to lead to a more effective representation of the grouping structure. One systematic effect of temporal grouping is on the relative frequency of positional confusions between immediately adjacent items: Under temporal grouping, adjacent transpositions that cross the group boundary are less frequent than those occurring within groups, while between-group transpositions relatively outnumber within-group transpositions for ungrouped lists (e.g., Maybery et al., 2002; Parmentier & Maybery, 2008). Maybery et al. (2002) found that extending the intergroup pause leads to increased accuracy for grouped lists, but does not modulate the pattern of positional confusion errors: The same relative dominance of within-group transpositions was observed, irrespective of the reduced overall number of errors with a longer intergroup pause (see the Appendix of Maybery et al., 2002). Supplementary materials for the present article report similar analyses of our Experiments 1 and 2, as well as of a third experiment in which only the intergroup pause was varied; these analyses reveal a similar lack of modulating effect of the intergroup pause on the difference between ungrouped and grouped lists in the relative frequencies of within- and between-group errors, despite an enhancing effect of intergroup duration on overall accuracy. This suggests that a hierarchical structure is put in place regardless of the intergroup pause, and that the additional time allows for some general strengthening of memory.

In the framework of contemporary models of serial recall, this strengthening of memory might come about in a number of ways. Rehearsal in the intergroup pause might offset decay, as suggested by Ryan (1969b) and consistent with the operation of rehearsal in models such as the primacy model (Page & Norris, 1998), the model of Burgess and Hitch (1999), and Henson’s (1998) Start–End Model. One problem for the primacy model is that it cannot account for the various effects of grouping on short-term memory, and recent research suggesting that decay is not the primary cause of forgetting from short-term memory (see, e.g., Lewandowsky & Oberauer, 2009; Lewandowsky, Oberauer, & Brown 2009; Oberauer & Lewandowsky, 2008) poses problems for all of these models. Alternatively, the additional time between groups might allow for the strengthening of associations between list items and their positional contexts (thereby reducing the relative effect of noise at recall) or the sharpening of positional representations coding within-group position (as assumed by Henson, 1998, to capture modality effects) and between-group position. However, one such mechanism that is ruled out by our results is the temporal gap between groups producing a linked increase in the distinctiveness of time-based representations coding the position of different groups, as in the OSCAR model (Brown et al., 2000). One prediction of this mechanism is that lengthening the pause between groups in the list will lead to a corresponding pause at output. However, the latency results presented here and elsewhere (Maybery et al., 2002) rule out this possibility. Indeed, Fig. 5 shows that lengthening the intergroup pause produces faster responding across the list. This again suggests that the processing taking place in the gap between groups enhances access to the contents of the groups.

Effects of sequence timing on the dynamics of recall

In contrast to the effects on recall accuracy, recall latencies appeared to be systematically affected by input timing at the level of individual items. A clear picture of the effect of input timing on recall latencies is seen in Fig. 4, in which a longer pause at input (squares) was associated with a longer interresponse duration in participants’ output. The one exception is seen at the group boundary (Serial Position 4) in the right column of Fig. 4, where a shorter input duration elicited either no difference or a longer pause at output. Along with the detrimental effects of a short intergroup duration on recall accuracy, it appears that a small gap at the group boundary slows initiation of recall of the second group. One reasonable interpretation is that some pointer to, or representation of, each group must be recalled prior to recall of the individual items within each group (e.g., Anderson & Matessa, 1997) and that these group representations are less accessible when less time is left at the group boundary for their encoding. The otherwise positive relationship between input and output timing was observed for both grouped and ungrouped lists in the ANOVAs associated with Fig. 4. The reliability of the effect for grouped lists is somewhat unclear: Experiment 1 and the data of Farrell (2008) failed to find an input–output timing relationship for grouped lists, although such an effect was apparent in Experiment 2. In any case, for ungrouped lists there is a clear and systematic relationship between input and output timing. A similar finding emerged from a study by Cowan, Elliott, Saults, Nugent, Bomb, and Hismjatullina (2006), who found that increasing the pace of list presentation also increased the pace of recall (where duration was varied between rather than within lists).

The mirroring of list timing in output dynamics (apart from at the group boundary) is of interest for two reasons. The first is that it demonstrates how input durations are represented in short-term memory. Along with other studies in which recall or recognition of the timing of sequences has been explicitly required (e.g., Collier & Logan, 2000; Farrell, 2008; Farrell & McLaughlin, 2007; Saito, 2001; Watkins et al., 1992), this shows that information about durations can be encoded, retained, and recalled from short-term memory. Although a viable candidate explanation for this ability would be the temporal representation of items’ occurrence in the oscillator-based OSCAR model (Brown et al., 2000), this explanation can be rejected on the basis of the lack of a TIE. Instead, a reasonable suggestion within the framework of current serial recall models is that some representation of the duration of individual items, along with other item information (e.g., phonological, lexical), is associated with their positional representations. Farrell (2008) found evidence consistent with this suggestion: When participants were asked to recall the timing of a list (by tapping a key in time with the remembered input sequence), there was a tendency to confuse durations close together on the list in some experiments.

As noted by Farrell (2008), even if people recalled duration information along with other aspects of items’ identity, from the perspective of most serial recall models there is no reason to reproduce this information in their output. Two explanations for this result are offered by production-based theories of short-term memory. In the ACT–R model of memory presented by Anderson, Bothell, Lebiere, and Matessa (1998), peaks in the latency function are produced by participants programming sub-sequences of output before producing each sub-sequence as a burst (cf. Sternberg, Monsell, Knoll, & Wright 1978). One explanation for the linkage between input and output dynamics is that participants use pauses in the input sequence to determine the point at which construction of a sub-sequence for output is halted and those items programmed into the subsequence are emitted in a burst. However, this begs the question of how participants remember the temporal structure of the input without this affecting response probabilities, a problem discussed with respect to the OSCAR model above. A second alternative is that serial recall is not subserved by a special short-term memory system, but instead reflects the operation of perception and motor processes, particularly speech production in the case of verbal materials (e.g., Jones, Macken, & Nicholls 2004; Macken & Jones, 2003). Timing has been suggested to play an important prosodic role in articulation and serial recall by cueing the hierarchical structuring of linguistic materials (Hughes, Marsh, & Jones 2009; cf. Frankish, 1989); however, the suggested role in the structuring of input sequences was not seen in our experiments, in that a TIE was absent at serial positions where an input–output relation in timing was observed. Nevertheless, several studies have shown the importance of timing for motor sequence learning: Variations in timing between repetitions of a sequence disrupt learning of that sequence, suggesting that relative timing is integrated into the action representation of a sequence (e.g., Shin & Ivry, 2002; Tubau et al., 2007). More generally, memory has been suggested to be the reenactment of presented events (Jones & Martin, 2009; Kent & Lamberts, 2008), which presumably includes information about the timing of those events. These alternative perspectives on cognition are of interest, but need further detailed specification in order to be rigorously compared with existing models of serial recall.

Conclusion

In finishing, we might ask how these various aspects of timing on serial recall could be accounted for under a single model. The results are generally consistent with existing models of serial recall that assume some multilevel representation of position in which the positional representations are event-based, not time-based, in order to account for the lack of a compelling TIE in serial recall data (Burgess & Hitch, 2006; Henson, 1998; Lewandowsky & Farrell, 2008). To explain the beneficial effects of an intergroup pause on accuracy, some process must operate to reduce errors on the list in a nonspecific fashion; some candidate mechanisms were outlined above. Finally, to explain the influence of input timing on output dynamics, as well as the ability of participants to explicitly recall the timing of sequences (e.g., Collier & Logan, 2000; Farrell, 2008; Watkins et al., 1992), it can be assumed that participants store information about the durations preceding or following individual items as part of the representation of the items themselves. The surprising manifestation of the temporal information in output dynamics—even when it is irrelevant—is surprising, but consistent with other work showing integration of information about dynamics into the representation of events and actions (e.g., Kent & Lamberts, 2008; Shin & Ivry, 2002; Tubau et al., 2007).