Multi-voxel pattern analysis of selective representation of visual working memory in ventral temporal and occipital regions
Highlights
► We studied spatial response patterns of visual cortical areas during working memory. ► Multivoxel pattern analysis reveals selective maintenance of face or scene pictures. ► Activity of fusiform and parahippocampal areas reflects the task relevant category. ► Activation patterns during selective maintenance and object recognition are similar.
Introduction
Studies of human and nonhuman primates have consistently shown that the ventral temporal and occipital regions are involved in the perception and recognition of visual stimuli (see review by Ungerleider and Haxby, 1994). These visual association regions in the posterior cortex show functional divisions specializing in categorical representation of objects such as faces, tools, words, etc. (e.g., Chao et al., 1999, Epstein and Kanwisher, 1998). It has been proposed that these regions are also involved in supporting visual working memory – the short-term representation of visual stimuli that are no longer physically available (Postle, 2006, Ranganath and D'Esposito, 2005). Neuroimaging findings, however, have been inconsistent thus far. Some showed that the inferior temporal region (e.g., the lateral fusiform gyrus) was active in tasks requiring holding faces (e.g., Druzgal and D'Esposito, 2003, Postle et al., 2003, Ranganath et al., 2004) and in tasks requiring refreshing recently seen faces (e.g., Johnson et al., 2007). Others, however, showed that the activity in the inferior temporal region was not long lasting (Jha and McCarthy, 2000) and subject to interference (Miller et al., 1993, Sreenivasan et al., 2007; but see Yoon et al., 2006 for different results).
Some investigators further examined the selectivity of the posterior visual association regions in representing specific visual working memory. Face and/or scene images were used as task stimuli in neuroimaging studies since the fusiform (FG) and parahippocampal gyri (PHG) are known to be more specialized in processing faces and scenes, respectively (e.g., Epstein and Kanwisher, 1998, Kanwisher et al., 1997). Participants were cued to remember a particular category of visual stimuli (e.g., Remember face but ignore scene, and vice versa), with the cue presented either prior to stimulus presentation for selective encoding (Gazzaley et al., 2005, Nobre et al., 2004) or after, for selective maintenance (Lepsien et al., 2005, Oh and Leung, 2010). Across studies, the PHG consistently showed elevated activity during selective encoding and selective maintenance of scene images. The FG, however, did not always show differential activity for selective processing of faces (compare: Gazzaley et al., 2005, Oh and Leung, 2010). A recent fMRI study reported that neither PHG nor FG was modulated by the number of face/scene images to be selectively maintained in working memory (Lepsien et al., 2011). Thus, it is unclear to what extent the different posterior association regions are involved in representing task-relevant visual working memory.
Most previous studies reviewed above applied univariate analysis to determine whether or not a brain region is activated while particular visual information is assumed to be held in working memory. Using multiple voxel pattern analysis (MVPA), recent studies successfully showed differential spatial patterns of activation in both striate and extrastriate areas for holding visual features (e.g., orientations, Harrison and Tong, 2009) and visual categories (e.g., faces, scenes and objects; Lewis-Peacock and Postle, 2008, Lewis-Peacock et al., 2012 [Experiment 1]). Through reanalyzing data from the second experiment of the Lewis-Peacock et al. (2012) study, Lewis-Peacock and Postle (2012) showed that their results on classification of task-relevant category (out of three potential categories: pseudowords, words, and line orientations) during the delay period were not affected even after excluding the suprathreshold voxels identified by the general linear model (GLM) as category-specific. Here, we further examined the activation patterns of the FG and PHG as well as other specific temporal/occipital regions in response to cued selective maintenance of task-relevant visual working memory in the presence of no-longer-relevant working memory.
We applied MVPA to previously published data (Oh and Leung, 2010) and conducted within-subject analysis to examine the activation patterns in the FG, PHG and other ventral temporal and occipital regions during selective maintenance of face/scene images. The task (Fig. 1A) comprised three phases: initial encoding (remembering two pictures, a face and a scene), selective maintenance (maintaining one of the two pictures according to a text cue), and recognition (judging whether the probe image is an exact match of the cued picture). We first trained and tested classifiers using activation patterns from the cue phase and examined classification performance across time during selective maintenance. In addition, we trained classifiers using activation patterns from the probe phase and from a separate localizer task, and tested these different classifiers on the cue-phase data to confirm that classification results for selective maintenance of faces/scenes are not due to the word cue itself. We were particularly interested in the FG and other ventral temporal and occipital regions involved in face processing since many of these regions did not show differential activity during selective maintenance in previous univariate analysis (see Fig. 1B).
Section snippets
Methods
We used the 12 datasets from a study published by Oh and Leung (2010). A detailed description of the experimental procedure and image preprocessing can be found in that paper. Here, we provide a brief summary on the task design and image acquisition and processing procedures.
Classification of activation patterns during probe recognition
To compare with previous findings, we first tested how accurately we could classify the probe category (face vs. scene) using activation patterns of the FG and ventral PHG from both hemispheres during the probe stage of the trials. Table 1 shows the average within-subject classification accuracies for classifiers trained and tested using fMRI data from the three time segments (early, middle and late) of the probe phase. Classification performance was well above chance (50%) for discriminating
Discussion
A primary feature of visual working memory is the flexible representation of visual information most relevant to the current task demand. We utilized MVPA to examine whether the spatial response patterns of the individually defined ventral temporal and occipital regions reflect the selected visual category in working memory. Although previously we found only differential level of activity in the PHG for selective representation of scene images and no significant effects for the FG, present
Summary
We applied MVPA to show that the spatial response patterns of specific posterior visual association regions including the fusiform gyrus and the parahippocampal gyrus carry information reflecting the task-relevant visual category in working memory. These findings of selective visual representation revealed by the multi-voxel activation patterns in the fusiform gyrus complemented our previous analysis which showed no differences in this region's average activity across the task conditions (Oh
Acknowledgments
We thank Juntian Shan for his help at the initial stage of this project. This work was partially supported by the State University of New York at Stony Brook including the office of the vice president of research the DIGITEO Subsample project, France, and NIH grants 1 R01 DA020949 and 1 R01 EB007530.
References (35)
- et al.
What fMRI has taught us about human vision
Curr. Opin. Neurobiol.
(1997) - et al.
A brief thought can modulate activity in extrastriate visual areas: top–down effects of refreshing just-seen visual stimuli
Neuroimage
(2007) - et al.
Directing spatial attention in mental representations: interactions between attentional orienting and working-memory load
Neuroimage
(2005) - et al.
Modulation of working-memory maintenance by directed attention
Neuropsychologia
(2011) - et al.
Decoding the internal focus of attention
Neuropsychologia
(2012) Low-level memory processes in vision
Trends Neurosci.
(2000)- et al.
Information mapping with pattern classifiers: a comparative study
Neuroimage
(2011) Working memory as an emergent property of the mind and brain
Neuroscience
(2006)- et al.
Seeking the neural substrates of visual working memory storage
Cortex
(2003) - et al.
Directing the minds eye: prefrontal, inferior and medial temporal mechanisms for visual working memory
Curr. Opin. Neurobiol.
(2005)