Elsevier

NeuroImage

Volume 166, 1 February 2018, Pages 185-197
NeuroImage

Beyond the word and image: II- Structural and functional connectivity of a common semantic system

https://doi.org/10.1016/j.neuroimage.2017.10.039Get rights and content

Highlights

  • Structural and functional connectivity for language and visual scene comprehension.

  • Structural tracks converge-diverge to the semantic anterior temporal site.

  • Parieto-temporal cortex is an integrative node between lateral and medial networks.

  • mLF integrity correlates with behavioural comprehension metric.

  • Neural activity is correlated in lateral and in medial nodes.

Abstract

Understanding events requires interplaying cognitive processes arising in neural networks whose organisation and connectivity remain subjects of controversy in humans. In the present study, by combining diffusion tensor imaging and functional interaction analysis, we aim to provide new insights on the organisation of the structural and functional pathways connecting the multiple nodes of the identified semantic system -shared by vision and language (Jouen et al., 2015). We investigated a group of 19 healthy human subjects during experimental tasks of reading sentences or seeing pictures. The structural connectivity was realised by deterministic tractography using an algorithm to extract white matter fibers terminating in the selected regions of interest (ROIs) and the functional connectivity by independent component analysis to measure correlated activities among these ROIs. The major connections link ventral neural stuctures including the parietal and temporal cortices through inferior and middle longitudinal fasciculi, the retrosplenial and parahippocampal cortices through the cingulate bundle, and the temporal and prefrontal structures through the uncinate fasciculus. The imageability score provided when the subject was reading a sentence was significantly correlated with the factor of anisotropy of the left parieto-temporal connections of the middle longitudinal fasciculus. A large part of this ventrally localised structural connectivity corresponds to functional interactions between the main parietal, temporal and frontal nodes. More precisely, the strong coactivation both in the anterior temporal pole and in the region of the temporo-parietal cortex suggests dual and cooperating roles for these areas within the semantic system. These findings are discussed in terms of two semantics-related sub-systems responsible for conceptual representation.

Introduction

Understanding the world constitutes a human aptitude that requires high-order semantic operations realized in distributed neural networks. A number of studies have been conducted to investigate the organisation of such semantic systems in healthy humans and neurological patients-for review see (Binder et al., 2009, Lambon Ralph et al., 2017). By neuroimaging meta-analyses (Binder et al., 2009), the existence of a large-scale network involved in language comprehension has been described in a set of left lateralized ventral regions widely distributed between the parietal, temporal and prefrontal cortices as well as the medial structures (parahippocampus, posterior cingulate, ventro-medial prefrontal cortex). By using verbal and non verbal tasks, multimodal integration or semantic processing have been investigated in patients (Bozeat et al., 2000, Jefferies and Lambon Ralph, 2006) and in normal subjects during fMRI (Vandenberghe et al., 1996, Visser and Lambon Ralph, 2011) and TMS (Pobric et al., 2010) explorations. In a recent report, we have identified a semantic network which was commonly activated during tasks of image and sentence comprehension (Jouen et al., 2015). Our findings suggest that the connections between inferior parietal cortex and anterior temporal cortex might play a key role in such a multimodal semantic representation. Accordingly, compelling evidence on a role of the temporal lobe in semantic processes has been provided by clinical observations in patients with anterior temporal lobes atrophy (Lambon Ralph, 2014, Patterson et al., 2007, Rice et al., 2015). These patients have a significant and progressive loss of semantic capabilities concerning linguistic as well as other sensory modalities (vision, auditory). Moreover, by diffusion-weighted imaging tractography, Binney et al., (2012) described how the temporal lobe may contain a caudo-rostral gradient of sensory features converging from modality specific encoding in the caudal regions to transmodal representation in the middle and rostral temporal regions. The authors suggest that corresponding bottom-up and top-down interactions should co-exist dynamically.

While the neural network underpinning semantic cognition has been well described especially in the language domain, the white matter connectivity responsible for the interregional communication among nodes of the network remains a matter of debate. In humans the traditional post-mortem fiber dissection allowed scientists to identify the major white matter fascicle trajectories with no precise termination of the fiber limits. More recently, the tractography technique based on diffusion tensor imaging (Le Bihan et al., 1986) offers an in vivo and noninvasive way to determine the associative pathways connecting the largely distributed brain regions. In the domain of semantic cognition, early tractography studies concerned the language system with the description of left lateralized parallel temporo-parieto-frontal pathways involved in speech production and comprehension (Catani et al., 2005, Glasser and Rilling, 2008, Hagmann et al., 2006, Hickok and Poeppel, 2004, Makris and Pandya, 2009, Powell et al., 2006). Such a two-route language connectivity includes (1) a direct dorsal temporo-frontal pathway made of the prominent arcuate fasciculus connecting Wernicke's area involved in speech comprehension and Broca's area involved in speech production, and (2) an indirect ventral pathway connecting the parietal lobe to Wernicke's area (posterior segment) and to Broca's area (anterior segment) (Catani et al., 2005, Parker et al., 2005, Saur et al., 2008). Consistent with the original post-mortem description, this left-lateralized white matter structural organisation of the language system has been further confirmed and developed by using -probabilistic DTI in normal or brain-damaged subjects (Binney et al., 2012, Ellmore et al., 2010, Fang et al., 2015, Floel et al., 2009, Han et al., 2013), -combined functional and anatomical connectivity (Saur et al., 2010) or -direct electrical stimulation for intraoperative mapping (Almairac et al., 2015, De Witt Hamer et al., 2011, Duffau et al., 2013). By combining anatomical and functional findings in human, Ueno et al., (2011) built a computationally-implemented model, of the dual pathways that provides a platform for simulation of language semantic processing.

Therefore, the dual stream model has been extended with functional descriptions suggesting multiple components of the ventral semantic pathways: the direct inferior fronto-occipital fasciculus (iFOF), the middle (mLF) and inferior (iLF) longitudinal fasciculi as well as the uncinate fasciculus (UF). The role of these different white matter bundles is not well defined in terms of their relative roles in semantic function. Based on the Hickok and Poeppel model (Hickok and Poeppel, 2004), the lexical-semantic area situated between Wernicke's area, primary auditory and visual associative cortex is likely implicated in associating multimodal information to concepts. The resulting semantic information should hence be conveyed to the anterior temporal pole and inferior frontal cortex across the middle temporal pathways as suggested in the literature (Glasser and Rilling, 2008, Jefferies, 2013, Jung et al., 2016, Lambon Ralph et al., 2017, Noonan et al., 2010, Saur et al., 2010). First, neuroimaging studies converge to the idea of complementary contributions in language comprehension of each one of these ventral tracts. The iLF might be involved in mediating lexical semantic information to be integrated and stored in the anterior temporal lobe (Saur et al., 2010, Wei et al., 2012), while the UF would be involved in modulating this conceptual representation by cognitive control mechanisms issued from the inferior prefrontal cortex (Harvey et al., 2013, Saur et al., 2010). Parallel and dorsal to the iLF, the mLF recently discovered in human constitutes a large tract of long-distance white matter fibers running in the superior temporal lobe from the posterior cortical regions to the anterior regions of the temporal lobe (Makris et al., 2009, Makris et al., 2017). Because of these large interregional connections, the mLF has been implicated in a variety of cognitive functions including language, visuospatial and attentional processes as well as high order multimodal association functions (Makris et al., 2017). Finally, based on a series of brain electrostimulation studies in patients, Duffau's team proposes the iFOF as the essential pathway for multimodal semantic processing (Almairac et al., 2015, Duffau et al., 2013, Moritz-Gasser et al., 2013). Like the mLF, the iFOF constitutes a long, multiregional association pathway running through the extreme capsule and connecting the occipital, parietal and postero-lateral temporal cortical areas to the prefrontal cortex including the orbito-frontal and dorsolateral regions (Catani and Thiebaut de Schotten, 2008, Martino et al., 2010). Based on their observations, Duffau and colleagues (Duffau et al., 2013) propose a dual stream model of the ventral connectivity constituted by (1) a direct multicomponent route, the iFOF involved in semantic knowledge, and (2) a secondary indirect route made of iLF and UF (and possibly the mLF), involved in compensable semantic processing. However, while the literature is converging on the structural description of the different ventral white matter pathways, their respective roles in semantic processing remain a matter of controversy, particularly in the domain of the connectivity underlying transmodal conceptualization.

We build on a previous fMRI neuroimaging research in healthy humans where we have identified a semantic network of neural structures commonly activated for pictures and sentences (Jouen et al., 2015). This network forms a ventral neural system largely distributed between lateral (parieto-temporal and prefrontal) and medial (prefrontal, cingulate and parahippocampal) cortical regions with emphasis on the role of the parieto-temporal region as a hub for heteromodal semantic operations. In the present study, by combining diffusion tensor imaging and functional interaction analysis, we aim to provide new insights on the organisation of the structural and functional pathways connecting the multiple nodes of the identified semantic system -shared by vision and language. Indeed, while the majority of studies investigating semantic network connectivity focus on language comprehension or on localized neural structures, we will attempt to characterize a global framework for semantic processing common to language and visual scenes and grounded on both neural and functional connectivity, including long range connections. In order to investigate such an organisation of structural and functional connections underlying an effective interregional communication, we used as seeds the regions of interest (ROIs) identified in the semantic network common to language and visual scene processing (Jouen et al., 2015). The anatomical and functional connectivity was explored respectively by deterministic tractography to isolate only the fibers terminating in the selected ROIs and by independent component analysis (ICA) to extract the significant correlations between activated areas.

Section snippets

Material and methods

The methods and the experimental paradigm have been previously described in an event-related fMRI investigation of conceptual processes shared between images and sentences (Jouen et al., 2015). The unique features of the methods for the novel analyses of this study are reported here.

Structural identification of the main fibers tracts

Among the fiber tracts defined with our 13 ROIs distributed between the two hemispheres, 12 tracts have been retained as they were found in at least 10 subjects (Table 2): 9 were identified in the left hemisphere, one in the medial structures between RSC and pH, and 2 were localized in the right hemisphere. Among 34 fiber tracts identified, 22 tracts were not retained for further analysis as each one was found in less than 10 subjects. The characteristics (percentage of fibers and factor of

Discussion

Combined with our previous work (Jouen et al., 2015), these new findings provide a general framework including activated sites, their anatomical connections and functional links involved in processing meaning of sentences and images. The structural pathways are constituted of ventral white matter (WM) tracts in both hemispheres. Lateral tracts link the parieto-temporal, middle and superior temporal and inferior prefrontal sites, and medial tracts link the ventro-medial frontal cortex to the

Conclusions

By providing complementary results on the activated sites and their connections during semantic analysis, this research yields a general framework of the functional processes and structural organisation underpinning semantic analysis. The major connections of white matter fibers link ventral neural stuctures including the parietal and temporal cortices through inferior and middle longitudinal fasciculi, the temporal and parahippocampal gyrus through the cingulate bundle, and the temporal and

Funding

This work was supported by the French National Research Agency (ANR Project “Comprendre” and “Spaquence”) and by the European Community (EU FP7 Project “Wysiwyd”).

Acknowledgements

The authors are grateful to Vincent Perlbarg for his helpful comments during data analysis with the Netbrainwork toolbox.

References (97)

  • F. Geranmayeh et al.

    Semantic retrieval during overt picture description: left anterior temporal or the parietal lobe?

    Neuropsychologia

    (2015)
  • D.Y. Harvey et al.

    Neuropsychological evidence for the functional role of the uncinate fasciculus in semantic control

    Neuropsychologia

    (2013)
  • G. Hickok et al.

    Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language

    Cognition

    (2004)
  • P. Hoffman et al.

    Differing contributions of inferior prefrontal and anterior temporal cortex to concrete and abstract conceptual knowledge

    Cortex; a J. Devoted study Nerv. Syst. Behav.

    (2015)
  • E. Jefferies

    The neural basis of semantic cognition: converging evidence from neuropsychology, neuroimaging and TMS

    Cortex; a J. Devoted Study Nerv. Syst. Behav.

    (2013)
  • A.L. Jouen et al.

    Beyond the word and image: characteristics of a common meaning system for language and vision revealed by functional and structural imaging

    NeuroImage

    (2015)
  • P.A. Lewis et al.

    Brain mechanisms for mood congruent memory facilitation

    NeuroImage

    (2005)
  • P.A. Lewis et al.

    Ventromedial prefrontal volume predicts understanding of others and social network size

    NeuroImage

    (2011)
  • G. Marrelec et al.

    Regions, systems, and the brain: hierarchical measures of functional integration in fMRI

    Med. image Anal.

    (2008)
  • J. Martino et al.

    Anatomic dissection of the inferior fronto-occipital fasciculus revisited in the lights of brain stimulation data

    Cortex; a J. Devoted study Nerv. Syst. Behav.

    (2010)
  • K. Meyer et al.

    Convergence and divergence in a neural architecture for recognition and memory

    Trends Neurosci.

    (2009)
  • N. Molinaro et al.

    Combinatorial semantics strengthens angular-anterior temporal coupling

    Cortex; a J. Devoted study Nerv. Syst. Behav.

    (2015)
  • J. Moll et al.

    Impairment of prosocial sentiments is associated with frontopolar and septal damage in frontotemporal dementia

    NeuroImage

    (2011)
  • S. Moritz-Gasser et al.

    Mapping the connectivity underlying multimodal (verbal and non-verbal) semantic processing: a brain electrostimulation study

    Neuropsychologia

    (2013)
  • T. Nichols et al.

    Valid conjunction inference with the minimum statistic

    NeuroImage

    (2005)
  • G. Northoff et al.

    Self-referential processing in our brain–a meta-analysis of imaging studies on the self

    NeuroImage

    (2006)
  • G.J. Parker et al.

    Lateralization of ventral and dorsal auditory-language pathways in the human brain

    NeuroImage

    (2005)
  • G. Pobric et al.

    Amodal semantic representations depend on both anterior temporal lobes: evidence from repetitive transcranial magnetic stimulation

    Neuropsychologia

    (2010)
  • H.W. Powell et al.

    Hemispheric asymmetries in language-related pathways: a combined functional MRI and tractography study

    NeuroImage

    (2006)
  • G. Rizzolatti et al.

    Premotor cortex and the recognition of motor actions

    Brain Res. Cognitive Brain Res.

    (1996)
  • D. Saur et al.

    Combining functional and anatomical connectivity reveals brain networks for auditory language comprehension

    NeuroImage

    (2010)
  • S.L. Thompson-Schill

    Neuroimaging studies of semantic memory: inferring “how” from “where”

    Neuropsychologia

    (2003)
  • T. Ueno et al.

    Lichtheim 2: synthesizing aphasia and the neural basis of language in a neurocomputational model of the dual dorsal-ventral language pathways

    Neuron

    (2011)
  • N.M. Zahr et al.

    Problem solving, working memory, and motor correlates of association and commissural fiber bundles in normal aging: a quantitative fiber tracking study

    NeuroImage

    (2009)
  • F. Almairac et al.

    The left inferior fronto-occipital fasciculus subserves language semantics: a multilevel lesion study

    Brain Struct. Funct.

    (2015)
  • L.W. Barsalou

    Grounded cognition

    Annu. Rev. Psychol.

    (2008)
  • P.J. Basser et al.

    In vivo fiber tractography using DT-MRI data

    Magnetic Reson. Med. Official J. Soc. Magnetic Reson. Med./Soc. Magnetic Reson. Med.

    (2000)
  • J.R. Binder et al.

    Toward a brain-based componential semantic representation

    Cogn. Neuropsychol.

    (2016)
  • J.R. Binder et al.

    Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies

    Cereb. Cortex

    (2009)
  • R.J. Binney et al.

    Convergent connectivity and graded specialization in the rostral human temporal lobe as revealed by diffusion-weighted imaging probabilistic tractography

    J. Cognitive Neurosci.

    (2012)
  • R.L. Buckner et al.

    The brain's default network: anatomy, function, and relevance to disease

    Ann. N. Y. Acad. Sci.

    (2008)
  • S.J. Carrington et al.

    Are there theory of mind regions in the brain? A review of the neuroimaging literature

    Hum. Brain Mapp.

    (2009)
  • M. Catani et al.

    Perisylvian language networks of the human brain

    Ann. Neurol.

    (2005)
  • L. Chen et al.

    A unified model of human semantic knowledge and its disorders

    Nat. Hum. Behav.

    (2017)
  • B. Crosson et al.

    Role of the basal ganglia in language and semantics: supporting cast

    Neural Basis Semantic Mem.

    (2007)
  • P.C. De Witt Hamer et al.

    Is the human left middle longitudinal fascicle essential for language? A brain electrostimulation study

    Hum. Brain Mapp.

    (2011)
  • H. Duffau

    Intraoperative cortico-subcortical stimulations in surgery of low-grade gliomas

    Expert Rev. Neurother.

    (2005)
  • H. Duffau et al.

    Toward a pluri-component, multimodal, and dynamic organization of the ventral semantic stream in humans: lessons from stimulation mapping in awake patients

    Front. Syst. Neurosci.

    (2013)
  • Cited by (15)

    • Common ERP responses to narrative incoherence in sentence and picture pair comprehension

      2021, Brain and Cognition
      Citation Excerpt :

      This was inspired by the groundbreaking work of Vandenberghe et al. (1996) in understanding the common semantic system, and extended their approach from simple images and single words, to rich images and full sentences. In further investigation of this hypothesis we performed detailed mapping of the white matter pathways and functional connectivity linking the cortical nodes of this distributed network, with major hubs in the anterior temporal cortex and the temporo-parietal cortex (Jouen et al., 2018). The dense connectivity of these areas highlights their roles in integrative processing.

    • The temporal pole: From anatomy to function—A literature appraisal

      2021, Journal of Chemical Neuroanatomy
      Citation Excerpt :

      Studies in healthy volunteers also confirmed that the temporal pole was a key area for semantic processing: Using a combination of diffusion tensor imaging and functional MRI, both anterior temporal poles were found to be a key component of a semantic system, shared by vision and language (Jouen et al., 2015, 2018) A follow-up of 306 elderly subjects without dementia showed that a semantic decline was specifically related to temporal pole atrophy (Pelletier et al., 2017).

    • The medial temporal written word processing system

      2019, Cortex
      Citation Excerpt :

      Perirhinal cortex also receives input from the parahippocampal gyrus, the dorsal bank of the polymodal superior temporal sulcus and the orbitofrontal cortex, and somatosensory input from the insula (Suzuki & Amaral, 1994). The long association fibre system that connects perirhinal cortex with the ventral occipitotemporal neocortex is the Inferior Longitudinal Fascicle (Jouen et al., 2018; Schmahmann & Pandya, 2006). Perirhinal cortex projects to a much wider extent of cortex than it receives input from (Suzuki & Naya, 2014).

    • Effects of Connectivity on Narrative Temporal Processing in Structured Reservoir Computing

      2022, Proceedings of the International Joint Conference on Neural Networks
    View all citing articles on Scopus
    View full text