Elsevier

NeuroImage

Volume 73, June 2013, Pages 8-15
NeuroImage

Multi-voxel pattern analysis of selective representation of visual working memory in ventral temporal and occipital regions

https://doi.org/10.1016/j.neuroimage.2013.01.055Get rights and content

Abstract

While previous results from univariate analysis showed that the activity level of the parahippocampal gyrus (PHG) but not the fusiform gyrus (FG) reflects selective maintenance of the cued picture category, present results from multi-voxel pattern analysis (MVPA) showed that the spatial response patterns of both regions can be used to differentiate the selected picture category in working memory. The ventral temporal and occipital areas including the PHG and FG have been shown to be specialized in perceiving and processing different kinds of visual information, though their role in the representation of visual working memory remains unclear. To test whether the PHG and FG show spatial response patterns that reflect selective maintenance of task-relevant visual working memory in comparison with other posterior association regions, we reanalyzed data from a previous fMRI study of visual working memory with a cue inserted during the delay period of a delayed recognition task. Classification of FG and PHG activation patterns for the selected category (face or scene) during the cue phase was well above chance using classifiers trained with fMRI data from the cue or probe phase. Classification of activity in other temporal and occipital regions for the cued picture category during the cue phase was relatively less consistent even though classification of their activity during the probe recognition was comparable with the FG and PHG. In sum, these findings suggest that the FG and PHG carry information relevant to the cued visual category, and their spatial activation patterns during selective maintenance seem to match those during visual recognition.

Highlights

► We studied spatial response patterns of visual cortical areas during working memory. ► Multivoxel pattern analysis reveals selective maintenance of face or scene pictures. ► Activity of fusiform and parahippocampal areas reflects the task relevant category. ► Activation patterns during selective maintenance and object recognition are similar.

Introduction

Studies of human and nonhuman primates have consistently shown that the ventral temporal and occipital regions are involved in the perception and recognition of visual stimuli (see review by Ungerleider and Haxby, 1994). These visual association regions in the posterior cortex show functional divisions specializing in categorical representation of objects such as faces, tools, words, etc. (e.g., Chao et al., 1999, Epstein and Kanwisher, 1998). It has been proposed that these regions are also involved in supporting visual working memory – the short-term representation of visual stimuli that are no longer physically available (Postle, 2006, Ranganath and D'Esposito, 2005). Neuroimaging findings, however, have been inconsistent thus far. Some showed that the inferior temporal region (e.g., the lateral fusiform gyrus) was active in tasks requiring holding faces (e.g., Druzgal and D'Esposito, 2003, Postle et al., 2003, Ranganath et al., 2004) and in tasks requiring refreshing recently seen faces (e.g., Johnson et al., 2007). Others, however, showed that the activity in the inferior temporal region was not long lasting (Jha and McCarthy, 2000) and subject to interference (Miller et al., 1993, Sreenivasan et al., 2007; but see Yoon et al., 2006 for different results).

Some investigators further examined the selectivity of the posterior visual association regions in representing specific visual working memory. Face and/or scene images were used as task stimuli in neuroimaging studies since the fusiform (FG) and parahippocampal gyri (PHG) are known to be more specialized in processing faces and scenes, respectively (e.g., Epstein and Kanwisher, 1998, Kanwisher et al., 1997). Participants were cued to remember a particular category of visual stimuli (e.g., Remember face but ignore scene, and vice versa), with the cue presented either prior to stimulus presentation for selective encoding (Gazzaley et al., 2005, Nobre et al., 2004) or after, for selective maintenance (Lepsien et al., 2005, Oh and Leung, 2010). Across studies, the PHG consistently showed elevated activity during selective encoding and selective maintenance of scene images. The FG, however, did not always show differential activity for selective processing of faces (compare: Gazzaley et al., 2005, Oh and Leung, 2010). A recent fMRI study reported that neither PHG nor FG was modulated by the number of face/scene images to be selectively maintained in working memory (Lepsien et al., 2011). Thus, it is unclear to what extent the different posterior association regions are involved in representing task-relevant visual working memory.

Most previous studies reviewed above applied univariate analysis to determine whether or not a brain region is activated while particular visual information is assumed to be held in working memory. Using multiple voxel pattern analysis (MVPA), recent studies successfully showed differential spatial patterns of activation in both striate and extrastriate areas for holding visual features (e.g., orientations, Harrison and Tong, 2009) and visual categories (e.g., faces, scenes and objects; Lewis-Peacock and Postle, 2008, Lewis-Peacock et al., 2012 [Experiment 1]). Through reanalyzing data from the second experiment of the Lewis-Peacock et al. (2012) study, Lewis-Peacock and Postle (2012) showed that their results on classification of task-relevant category (out of three potential categories: pseudowords, words, and line orientations) during the delay period were not affected even after excluding the suprathreshold voxels identified by the general linear model (GLM) as category-specific. Here, we further examined the activation patterns of the FG and PHG as well as other specific temporal/occipital regions in response to cued selective maintenance of task-relevant visual working memory in the presence of no-longer-relevant working memory.

We applied MVPA to previously published data (Oh and Leung, 2010) and conducted within-subject analysis to examine the activation patterns in the FG, PHG and other ventral temporal and occipital regions during selective maintenance of face/scene images. The task (Fig. 1A) comprised three phases: initial encoding (remembering two pictures, a face and a scene), selective maintenance (maintaining one of the two pictures according to a text cue), and recognition (judging whether the probe image is an exact match of the cued picture). We first trained and tested classifiers using activation patterns from the cue phase and examined classification performance across time during selective maintenance. In addition, we trained classifiers using activation patterns from the probe phase and from a separate localizer task, and tested these different classifiers on the cue-phase data to confirm that classification results for selective maintenance of faces/scenes are not due to the word cue itself. We were particularly interested in the FG and other ventral temporal and occipital regions involved in face processing since many of these regions did not show differential activity during selective maintenance in previous univariate analysis (see Fig. 1B).

Section snippets

Methods

We used the 12 datasets from a study published by Oh and Leung (2010). A detailed description of the experimental procedure and image preprocessing can be found in that paper. Here, we provide a brief summary on the task design and image acquisition and processing procedures.

Classification of activation patterns during probe recognition

To compare with previous findings, we first tested how accurately we could classify the probe category (face vs. scene) using activation patterns of the FG and ventral PHG from both hemispheres during the probe stage of the trials. Table 1 shows the average within-subject classification accuracies for classifiers trained and tested using fMRI data from the three time segments (early, middle and late) of the probe phase. Classification performance was well above chance (50%) for discriminating

Discussion

A primary feature of visual working memory is the flexible representation of visual information most relevant to the current task demand. We utilized MVPA to examine whether the spatial response patterns of the individually defined ventral temporal and occipital regions reflect the selected visual category in working memory. Although previously we found only differential level of activity in the PHG for selective representation of scene images and no significant effects for the FG, present

Summary

We applied MVPA to show that the spatial response patterns of specific posterior visual association regions including the fusiform gyrus and the parahippocampal gyrus carry information reflecting the task-relevant visual category in working memory. These findings of selective visual representation revealed by the multi-voxel activation patterns in the fusiform gyrus complemented our previous analysis which showed no differences in this region's average activity across the task conditions (Oh

Acknowledgments

We thank Juntian Shan for his help at the initial stage of this project. This work was partially supported by the State University of New York at Stony Brook including the office of the vice president of research the DIGITEO Subsample project, France, and NIH grants 1 R01 DA020949 and 1 R01 EB007530.

References (35)

  • T.V. Sewards

    Neural structures and mechanisms involved in scene recognition: a review and interpretation

    Neuropsychologia

    (2011)
  • L.G. Ungerleider et al.

    “What” and “where” in the human brain

    Curr. Opin. Neurobiol.

    (1994)
  • J.H. Yoon et al.

    Differential effects of distraction during working memory on delay-period activity in the prefrontal cortex and the visual association cortex

    Neuroimage

    (2006)
  • M.J. Arcaro et al.

    Retinotopic organization of human ventral visual cortex

    J. Neurosci.

    (2009)
  • L.L. Chao et al.

    Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects

    Nat. Neurosci.

    (1999)
  • T.J. Druzgal et al.

    Dissecting contributions of prefrontal cortex and fusiform face area to face working memory

    J. Cogn. Neurosci.

    (2003)
  • R. Epstein et al.

    A cortical representation of the local visual environment

    Nature

    (1998)
  • Cited by (0)

    View full text