Neural representation of stimulus-response associations during task preparation
Introduction
Environmental context guides our behavior and biases our cognitive processing (Hazeltine and Schumacher, 2016, Schumacher and Hazeltine, under review). For example, we may plan to make either a sandwich or a hamburger for dinner. If looking in the breadbox reveals that we have hamburger buns, this information allows us to select the task file for making hamburgers. In this way, the additional environmental cue (the contents of the breadbox) allows us to adjust our behavior based on knowledge of our available resources. Laboratory research supports the idea that we can use partially informative cues to modify and facilitate behavioral responses dynamically during decision making (Miller, 1982, Rosenbaum, 1983). In other words, actions can be partially planned such that some parts of a future action or action sequence are specified, but other parts are based on future stimuli. We may plan to make a burger after looking in the breadbox, but we must leave many other actions (e.g. how to flavor the beef, how to cook the patties, etc.) unspecified until we gather additional information from the environment (e.g., what are the available spices, is the grill working, etc.).
One area of research has been identifying the locus of the cue-preparation benefit. Miller (1982) conducted a series of experiments investigating this issue. In his procedure, stimuli consisted of four crosses on the screen that were spatially mapped to four buttons on a response box that were mapped to the index and middle fingers on each hand in order from left to right (i.e., leftmost cross mapped to left middle finger, second cross to left index, and so on). For each trial, the participants saw a warning signal that showed all four crosses, followed by a cue signal which consisted of all four crosses (uninformative) or a subset of two (informative). The subset of two crosses could indicate any two of the four positions; that is, any two fingers could be indicated for the upcoming response. The uninformative cue indicated all four fingers for the potential response. Participants were finally presented with a single cross in one of the four possible positions, at which point they pressed the corresponding button. Unsurprisingly, participants were faster to respond when the cues were informative than when they were uninformative. However, not all informative cues produced equal benefits; cuing two responses on the same hand produced shorter RTs than cuing two responses on the same finger (i.e., an index or a middle finger response). Miller proposed that information was passed in discrete quanta to the motor system as it was made available; that is, information about a response that allows for the preparation of one salient subset of responses over another is processed as it arrives, reducing the remaining processing required upon the presentation of the stimulus. To explain the difference in the benefit of different types of cues, Miller proposed that the structure of the motor system affords a hand advantage because this information can be extracted and the response subset prepared more quickly than finer-grained information (see also, Rosenbaum (1980)).
Reeve and Proctor, 1984, de Jong et al., 1988 proposed a different explanation. They argued that the hand advantage found in Miller's (1982) results was due, not to a response preparation advantage, but to the spatial correspondence between the visual cue positions and the associated response mapping. They showed that removing this stimulus-response correspondence in turn reduced the hand advantage. Specifically, they had participants complete Miller's task, but with their hands positioned to overlap one another so that the leftmost cross corresponded to the left middle finger, the second cross to the right index finger, the third cross to the left index finger, and the fourth cross to the right middle finger. In this position, cues indicating responses on the left hand (i.e., crosses shown in the first and third positions) were no longer presented in the left side of the screen. Proctor and Reeve posited that the cuing effect takes place in response selection. They proposed that the correspondence between the stimulus and response allows participants to translate a cue into a subset of potential pairs from which the response will be selected. Removing the visual-anatomical correspondence afforded by Miller's design limits the utility of the cue.
In parallel with this debate about the information processing locus of the cuing benefit, measures of brain activity have been used to investigate the neural correlates of cue-preparation benefits. At the response level, event-related potentials (ERPs) during cue preparation and have demonstrated preparatory activity in motor regions (Leuthold et al., 1996). Leuthold and colleagues used the lateralized readiness potential (LRP), a measure of the difference in activity between electrodes above both the left and right motor cortex. They found that the LRP increased to a cue indicating which hand would be required to produce the upcoming response, suggesting that motor regions may begin to prepare a response even when the actual digit necessary for the response remains unknown. There is also a wealth of data showing that sensory regions respond to cues, especially from investigations of selective attention. For example, many studies show that a cue indicating a relevant upcoming stimulus dimension increases activity in sensory regions that process the cued location or dimension. This increase in activity is associated with a corresponding facilitation in stimulus processing (for reviews see Desimone and Duncan, 1995 and Kastner and Ungerleider, 2000).
Thus, research shows both an early (stimulus) and late (response) effect on processing of a cue in the regions that process those types of information. However, the interaction between these effects has received less investigation. Here, the cognitive literature may lead to predictions about the relationship between stimulus- and response-level influences in the brain. Returning to the response cuing literature, Miller (1982) posited that locus of the cueing effect is in response preparation. If this is the case, motor preparatory activity during cue presentation should be independent of stimulus-related activity – that is, information contained in a cue should result in activity differences only in the region that processes the information directly indicated by the cue. For example, a cue for stimulus color should result in modulation only in V4. On the other hand, Reeve and Proctor (1984) posited that the cueing effect occurred in response selection – where responses are associated with stimuli. If this is the case, then one might expect not only that a cue will influence the regions that process that information, but also that there may be activity across both the stimulus- and response-related regions related to executing the indicated task.
A more recent cognitive theory that is consistent with the interaction between stimulus and response processing in cue preparation is the grouping model (Adam et al., 2003b, Adam et al., 2005). According to this account, the presentation of a cue initiates grouping processes that act on both stimulus and response representations. For example, in Miller's (1982) design, stimuli could occur in one of four spatial locations, which were mapped to the first two fingers of each hand in spatial order. The grouping model posits that participants group the stimuli into left and right hemifields, and group responses anatomically by hand. Cuing for either the left two or right two stimuli indicates salient groups at both the stimulus and response levels, resulting in a behavioral benefit. In this way, the model holds that cue information allows for the preparation of a salient set of stimulus-response pairs.
The grouping process described by Adam and colleagues (Adam et al., 2003a) results in the formation of a task file (Schumacher and Hazeltine), in which the scope of the possible stimulus and response features for a given task are bound together into associated pairs along with motivational and other contextual information that allow participants to perform the task. When preparing to perform a task, the relevant task file is activated, and actions are coordinated according to the associations within the active task file.
These task files may provide a cognitive mechanism for the complex pattern of behaviors observed in response cuing. Specifically, participants may use the grouping process to link salient subsets of the task into separate task files, with additional bound context (i.e., the relevant cue) for when to select each subset. Then, when participants are given one of the relevant cues on a given trial, they prepare the task file indicated by that cue, and execute the task according to the associations represented within that task file.
In the brain, cue-related activity may represent the preparation of these task files in anticipation of the task. Adam et al. (2003a) used fMRI to investigate this process. This study used Miller's (1982) design with consistent or inconsistent S-R mappings and compared blocks of cued activity to uncued activity, which allowed them to separate activity due to informative versus uninformative cues. They found activation in a number of regions relating to cued activity, including prefrontal cortex (PFC, including middle frontal gyrus, MFG; dorsal and lateral premotor cortex, DPMC/LPMC; supplementary motor area, SMA), parietal cortex (intra-parietal sulcus, IPS; superior parietal cortex, SPC; inferior parietal cortex, IPC) and basal ganglia. These regions, then, are specifically related to the processing and implementation of the information contained within a cue. Notably, a number of these regions are specifically related to stimulus and response processing (e.g., parietal cortex processes spatial information of stimuli).
The pattern of brain activity to the cue found by Adam et al. (2003a) closely corresponds to regions associated with response selection processes. Schumacher et al. (2003) used two choice-reaction tasks to investigate the neural correlates of spatial and non-spatial response selection. For each task, the authors varied the number of possible stimulus-response pairs on a given trial using a precue that indicated some subset of the available options. fMRI data recorded during the performance of each task showed distinct regions of activation in parietal, temporal, and frontal cortices for spatial versus non-spatial tasks. The frontal activity corresponded with premotor regions, which are involved in motor response preparation. The parietal and temporal activity, on the other hand, corresponded to regions involved in stimulus processing. Moreover, the activity in parietal cortex was more dorsal for the spatial task and more ventral for the non-spatial, consistent with previous research that has described a similar division of stimulus processing along these lines (Ungerleider et al., 1998). These results lend support to the idea that cuing effects may in fact reflect preparation in the specific processing region(s) involved in executing the upcoming task corresponding to the relevant stimuli and responses (Adam et al., 2003b, Adam et al., 2005, Reeve and Proctor, 1984).
Many of the regions identified by Adam et al. (2003a) have been proposed to mediate a wide range of cognitive control processes (for review, see Fuster, 2001, Miller and Cohen, 2001), so the assumption that they play a role in biasing attention to S-R pairs is consistent with its putative role mediating cognitive control. However, the method employed in that study did not allow for the isolation of activity for the different task files that Schumacher and Hazeltine would predict to be driving behavior on each trial; thus, the exact nature of the biasing mechanisms and how control affects stimulus and response processing remains unknown. If, as Miller (1982) proposed, the cue benefit occurs during response preparation, then the cue benefit may operate independently on stimulus- and response-related processing regions. On the other hand, if the cue benefit is the result of the response selection processes posited by Proctor and Reeve, then one might expect both downstream (response) and upstream (stimulus) effects of either a stimulus and/or response cue, consistent with a response selection locus (Reeve and Proctor, 1984). In this latter case, there are two possible instantiations of this process: either the cue may result in nonspecific preparatory processes that cover the full scope of the task, regardless of the information contained in the cue; or preparation may operate within the scope of the indicated task file, resulting in a bias of activity only in regions involved in processing that subset of the task.
To investigate this, we employed event-related fMRI during a response-cuing task. Participants learned stimulus-response mappings in which images of faces and places were mapped separately to the left and right hands, and were instructed to respond with the corresponding button press each time a picture appeared on screen. On some trials, a cue presented before the stimulus told the participant either what type of picture (face or place) would be presented or what hand (left or right) would be used to make the response. Importantly, the separation of stimulus type by hand allowed participants to represent the task as two subtasks, each associated with a unique stimulus type and response hand. Additionally, this meant that any informative cue implicitly indicated both a stimulus type and response hand, regardless of what specific information was imparted by the cue. The trial structure and mapping are illustrated in Fig. 1.
Motor responses are mediated by contralateral motor cortical regions and ipsilateral cerebellar regions (Kandel et al., 2000). Previous literature has also demonstrated the existence of regions specialized for both face (viz., fusiform face area, FFA; Kanwisher et al., 1997, Kanwisher and Yovel, 2006) and place (parahippocampal place area, PPA; Epstein and Kanwisher, 1998) processing. We made use of this known functional architecture to investigate the interaction between stimulus and response cuing effects. To investigate the specific effects of specific stimulus- and response-informative cues on brain regions previously identified as showing cuing effects, we compared activity in the regions of interest (ROIs) for stimulus- and response-specific cue types (i.e., face versus place stimulus cues and left versus right response cues); these ROIs are visualized in Fig. 2. To understand how the cue affected activity in these stimulus and response ROIs, we investigated two comparisons. The explicit cue comparison was based on the stimulus or response that was directly referenced by the cue. For example, for a participant whose mapping involved making left hand responses to face stimuli and right hand responses to place stimuli, an F presented during the cue period explicitly cued an upcoming face stimulus (and therefore FFA). The implicit cue comparison was based on the stimulus or response that was indirectly indicated by a cue. In the previous example, the same F cue, while explicitly cuing a face stimulus, also indirectly indicated that the upcoming response required the left hand (and therefore right PMC and other response-related regions).
If stimulus and response cues result in independent modulation of their respective stimulus or response regions, this supports a peripheral locus (sensory or response preparation processes) of the cuing benefit. If instead, the cuing effect has a response selection locus, two outcomes are possible. If the task is simply represented as a single task file (across all S-R associations), then we may see a general activation of all of the task-related regions on every trial, regardless of the cue information. These regions should furthermore show mutual connectivity across cue types. If, on the other hand, stimulus type and response hand are bound together in a representation specific to the subtasks (face or place stimuli), then cuing one dimension (e.g. ‘faces’) may cause increased activity in brain areas recruited for both the indicated dimension as well as the associated dimension (e.g. ‘left hand’), even though the link to the associated dimension is implicit. Additionally, we predict that these regions may show increased connectivity between regions that process the relevant pairs of stimulus type and response hand as a function of the mapping.
Section snippets
Behavioral results
All analyses were conducted on the data from Session 2, error bars in all graphs represent the standard error of the relevant effect. Accuracies approached ceiling (Overall average=94.8%) and were analyzed using a modified arcsine transformation (, corrected for ceiling effects; Sheskin, 2003). A two-factor, repeated measures ANOVA for cue type and CSI duration showed no main effect of cue type, F(2,48)=.52, p=.600, but did show a main effect of CSI, F(2,48)=5.80, p=.006, as well as a
Discussion
The present experiment used an event-related response-cuing procedure to investigate brain activity to stimulus and response cues to identify how S-R pairings affect processing. Our experiment used a response cuing design in which participants learned a mapping that could be represented as two task files associating each response hand with a unique stimulus type. Cues given at the start of each trial could give no information, stimulus information, or response information. Informative cues then
Participants
Participants included 44 volunteers from the Georgia Institute of Technology community between the ages of 18 and 38 years old (15 female, 28 male). Six participants (3 female, 3 male) withdrew from the study before completion of both sessions; an additional 12 (5 female, 7 male) were not included in the analyses due to performance issues (1 sleeping, 1 not responding to stimuli) or excess motion (repeated translations of greater than 1 mm across a single block; position changes were measured in
References (42)
Rapid visuomotor preparation in the human brain: a functional MRI study
Brain Res. Cognit. Brain Res.
(2003)Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes
Trends Cogn. Sci.
(2008)- et al.
The functional role of cross-frequency coupling
Trends Cognit. Sci.
(2010) AFNI: software for analysis and visualization of functional magnetic resonance neuroimages
Comput. Biomed. Res. Int. J.
(1996)- et al.
Common regions of the human frontal lobe recruited by diverse cognitive demands
Trends Neurosci.
(2000) Analysis of fMRI time-series revisited
NeuroImage
(1995)The prefrontal cortex--an update: time is of the essence
Neuron
(2001)- et al.
Becoming a “Greeble” expert: exploring mechanisms for face recognition
Vis. Res.
(1997) - et al.
Separating processes within a trial in event-related functional MRI
Neuroimage
(2001) - et al.
Separating processes within a trial in event-related functional MRI
Neuroimage
(2001)
The movement precuing technique: assumptions, applications, and extensions
Adv. Psychol.
Analysis of fMRI time-series revisited—again
NeuroImage
Preparing for perception and action (I): the role of grouping in the response-cuing paradigm
Cognit. Psychol.
Preparing for perception and action (II): automatic and effortful processes in response cueing
Vis. Cogn.
The frontopolar cortex and human cognition: evidence for a rostrocaudal hierarchical organization within the human prefrontal cortex
Psychobiology
On some functions of the human prefrontal cortexa
Ann. N. Y. Acad. Sci.
Neural mechanisms of selective visual attention
Annu Rev. Neurosci.
Experimental approach to prefrontal functions in humans
Ann. N. Y. Acad. Sci.
A cortical representation of the local visual environment
Nature
The neural mechanisms of top-down attentional control
Nat. Neurosci.
Cited by (12)
Repetition priming in amnesia: Distinguishing associative learning at different levels of abstraction
2019, NeuropsychologiaCitation Excerpt :Therefore, we suggest that observations of impaired within-task priming in amnesia in a previous study (Schnyer et al., 2006) and in the current task likely reflect binding deficits at the task level. Indeed, it is well established that associations can form between stimuli and the task sets under which they are processed (e.g., “task files”), and that stimulus-task bindings make important contributions to priming and action selection (Cookson et al., 2016; Crump and Logan, 2010; Waszak et al., 2003, 2004; Wylie and Allport, 2000). Notably, a recent neuroimaging study found evidence for hippocampal involvement in the binding of stimuli to more abstract task representations and attentional control states during sequential (short-lag) priming (Jiang et al., 2015).
The Binary Structure of Event Files Generalizes to Abstract Features: A Nonhierarchical Explanation of Task Set Boundaries for the Congruency Sequence Effect
2022, Journal of Experimental Psychology: Learning Memory and CognitionRethinking attentional reset: Task sets determine the boundaries of adaptive control
2022, Quarterly Journal of Experimental PsychologyDissociating the Neural Correlates of Planning and Executing Tasks with Nested Task Sets
2022, Journal of Cognitive NeuroscienceTask structure boundaries affect response preparation
2020, Psychological Research