Elsevier

NeuroImage

Volume 54, Issue 2, 15 January 2011, Pages 875-891
NeuroImage

Network modelling methods for FMRI

https://doi.org/10.1016/j.neuroimage.2010.08.063Get rights and content

Abstract

There is great interest in estimating brain “networks” from FMRI data. This is often attempted by identifying a set of functional “nodes” (e.g., spatial ROIs or ICA maps) and then conducting a connectivity analysis between the nodes, based on the FMRI timeseries associated with the nodes. Analysis methods range from very simple measures that consider just two nodes at a time (e.g., correlation between two nodes' timeseries) to sophisticated approaches that consider all nodes simultaneously and estimate one global network model (e.g., Bayes net models). Many different methods are being used in the literature, but almost none has been carefully validated or compared for use on FMRI timeseries data. In this work we generate rich, realistic simulated FMRI data for a wide range of underlying networks, experimental protocols and problematic confounds in the data, in order to compare different connectivity estimation approaches. Our results show that in general correlation-based approaches can be quite successful, methods based on higher-order statistics are less sensitive, and lag-based approaches perform very poorly. More specifically: there are several methods that can give high sensitivity to network connection detection on good quality FMRI data, in particular, partial correlation, regularised inverse covariance estimation and several Bayes net methods; however, accurate estimation of connection directionality is more difficult to achieve, though Patel's τ can be reasonably successful. With respect to the various confounds added to the data, the most striking result was that the use of functionally inaccurate ROIs (when defining the network nodes and extracting their associated timeseries) is extremely damaging to network estimation; hence, results derived from inappropriate ROI definition (such as via structural atlases) should be regarded with great caution.

Research Highlights

► We evaluate many methods for estimating brain networks from FMRI data ► Covariance-based methods are the most successful at finding network edges ► Finding network directionality is harder; Patel's tau performs the best ► Lag-based methods (e.g. Granger) perform badly ► The use of incorrect ROIs (e.g. from structural atlases) is dangerous

Introduction

Neuroimaging is used to study many aspects of the brain's function and structure; one area of rapidly increasing interest is the mapping of functional networks. Such mapping typically starts by identifying a set of functional “nodes”, and then attempts to estimate the set of connections or “edges” between these nodes. In some cases, the directionality of these connections is estimated, in an attempt to show how information flows through the network.

There are many ways to define network nodes. In the case of electrophysiological data, the simplest approach is to either consider each recorded channel as a node, or instead use spatial sources after source reconstruction (spatial localisation) has been carried out. In the case of FMRI, nodes are often defined as spatial regions of interest (ROIs), for example, as obtained from brain atlases or from functional localiser tasks. Alternatively, independent component analysis (ICA) can be run to define independent components (spatial maps and associated timecourses), which can be considered network nodes, although the extent to which this makes sense depends on the number of components extracted (the ICA dimensionality). If a low number of components is estimated (Kiviniemi et al., 2003), then it makes more sense to think of each component itself as a network. This will often include several non-contiguous regions, all having the same timecourse according to the ICA model, and hence within-component network analysis is not possible without further processing, such as splitting the components and re-estimating each resulting node's timeseries. Furthermore, between-component network analysis is quite possibly not reasonable, as each component will in itself constitute a gross, complex functional system. However, if a higher number of components is estimated (Kiviniemi et al., 2009), these are more likely to be smaller, isolated regions (functional parcels), which can more sensibly be then considered as nodes for use in network analysis.

Once the nodes are defined, each has its own associated timecourse. These are then used to estimate the connections between nodes—in general, the more similar the timecourses are between any given pair of nodes, the more likely it is that there is a functional connection between those nodes. Of course, correlation (between two timeseries) does not necessarily imply either causality (in itself it tells one nothing about the direction of information flow), or whether the functional connection between two nodes is direct (there may be a third node “in-between” the two under consideration, or a third node may be feeding into the two, without a direct connection existing between them). This distinction between apparent correlation and true, direct functional connection (sometimes referred to as the distinction between functional and effective connectivity respectively; Friston, 1994) is very important if one cares about correctly estimating the network. For example, in a 3-node network where A B C, and with external inputs (or at least added noise that feeds around the network) for all nodes, then all three nodes' timeseries will be correlated with each other, so the “network estimation method” of simple correlation will incorrectly estimate a triangular network. However, another simple estimation method, partial correlation, can correctly estimate the true network; this works by taking each pair of timeseries in turn, and regressing out the third from each of the two timeseries in question, before estimating the correlation between the two. If B is regressed out of A and C, there will no longer be any correlation between A and C, and hence the spurious third edge of the network (AC) is correctly eliminated.

The question of directionality is also often of interest, but in general is harder to estimate than whether a connection exists or not. For example, many methods, such as the two mentioned above (full correlation and partial correlation) give no directional information at all. The methods that do attempt to estimate directionality fall into three general classes. The first class is “lag-based”, the most common example being Granger causality (Granger, 1969). Here it is assumed that if one timeseries looks like a time-shifted version of the other, then the one with temporal precedence caused the other, giving an estimation of connection directionality. The second class is based on the idea of conditional independence, and generally starts by estimating the (zero-lag) covariance matrix between all nodes' timeseries (hence such methods are based on the same raw measure of connectivity as correlation-based approaches—but attempt to go further in utilising this matrix to draw more complex inferences about the network). Such methods may look at the probability of pairs of variables conditional on sets of other variables; for example, Bayes net methods (Ramsey et al., 2010) in general estimate directionality by first orienting “unshielded colliders” (paths of the form A B C) and then drawing inferences based on algorithm-specific assumptions regarding what further orientations are implied by these colliders. The third class of methods utilises higher order statistics than just the covariance; for example, Patel's pairwise conditional probability approach (Patel et al., 2006) looks at the probability of A given B, and B given A, with asymmetry in these probabilities being interpreted as indicating causality.

A large number of network estimation methods have been used in the neuroimaging literature, with varying degrees of validation. The closer a given modality's data is to the underlying neural sources, the simpler it is to interpret the data and analyses resulting from it. In the case of FMRI, the data is a relatively indirect measure of the neural activity, being distanced from the underlying sources by many confounding stages, particularly the nonlinear neuro-vascular coupling that adds (generally unknown amounts of) significant blurring and delay to the neural signal (Buxton et al., 1998). This means that very careful validation is necessary before network estimation methods applied to FMRI data can be safely interpreted, and, unfortunately, it is too often the case that careful, sufficiently rich, validation is not carried out before real data is analysed and interpreted. Several approaches have been applied to electrophysiological data, and have been well validated for that application domain; however, because FMRI data is so much further removed from the underlying sources of interest than is generally the case with the various electrophysiological modalities, FMRI-specific validations are of particular importance. We concentrate solely on FMRI data in this paper. We simulate resting FMRI data, although the results will also in general be relevant for task FMRI (in fact, the input timings generated in the simulations could equally be viewed as simulating an event-related task FMRI experiment).

The purpose of this work is to apply a rich biophysical FMRI model to a range of network scenarios, in order to provide a thorough simulation-based evaluation of many different network estimation methods. We have compared their relative sensitivities to finding the presence of a direct network connection, their ability to correctly estimate the direction of the connection, and their robustness against various problems that can arise in real data. We find that some of the methods in common use are not effective approaches, and even can easily give erroneous results.

Section snippets

Methods: Simulations

Networks of varied complexity were used to simulate rich, realistic BOLD timeseries. The simulations were based upon the dynamic causal modelling (DCM; Friston et al., 2003) FMRI forward model, which uses the nonlinear balloon model (Buxton et al., 1998) for the vascular dynamics, sitting on top of a neural network model. We now describe in detail how our simulations were generated. Specific simulation parameters given are true in general for most of the evaluations, except where particular

Methods: Network modelling methods tested

We now give a brief description of each of the methods tested. Where minor variants of each main method (including alternative choices in controlling parameters) performed universally worse than other variants, we exclude the unsuccessful variants from further consideration in the paper, in order to maximise the clarity of presentation. We describe all variants tested (including descriptions of those that were rejected) within this section.

Results

We begin by explaining how we summarised the outputs from testing the different network modelling approaches. (For a summary of the specifications for all 28 simulations see Table 1.) Results from one of the most “typical” network scenarios (Sim2) are shown in Fig. 3, which has 10 nodes, 10 min FMRI sessions for each subject, TR = 3 s, measurement noise (thermal noise added onto the BOLD signal) of 1%, and HRF lag variability of ± 0.5 s.

For some of these plots we use the raw connection strength

Discussion

Although some of the different data scenarios generated quite variable sets of results, a general picture does emerge, that should be applicable across a large fraction of real FMRI experiments.

With respect to estimating the presence of a network connection, the overall results suggest that the “Top-3” (Partial correlation, ICOV and the Bayes net methods) often perform excellently, with a sensitivity of more than 90% on “typical” data. The Bayes net methods are as good as, or slightly better

Acknowledgments

We are very grateful to: Gopikrishna Deshpande for helpful discussions, Aslak Grinsted for providing the Crosswavelet and Wavelet Coherence toolbox, Chandler Lutz for providing the pairwise Granger causality code, Alard Roebroeck for helpful discussions, Alois Schlögl for providing the BioSig toolbox, Mark Schmidt for providing the regularised inverse covariance code (and for helpful discussions), Anil Seth for providing the Causal Connectivity Analysis toolbox (and for helpful discussions),

References (59)

  • M. Kamiński et al.

    Topographic analysis of coherence and propagation of EEG activity during sleep and wakefulness

    Electroencephalogr. Clin. Neurophysiol.

    (1997)
  • V. Kiviniemi et al.

    Independent component analysis of nondeterministic fMRI signal sources

    Neuroimage

    (2003)
  • G. Marrelec et al.

    Partial correlation for functional brain interactivity investigation in functional MRI

    Neuroimage

    (2006)
  • E. Pereda et al.

    Nonlinear multivariate analysis of neurophysiological signals

    Prog. Neurobiol.

    (2005)
  • J. Ramsey et al.

    Six problems for causal inference from fMRI

    Neuroimage

    (2010)
  • A. Roebroeck et al.

    Mapping directed influence over the brain using Granger causality and fMRI

    Neuroimage

    (2005)
  • B.P. Rogers et al.

    Functional MRI and multivariate autoregressive models

    Magnetic Resonance Imaging

    (2010)
  • A.K. Seth

    A MATLAB toolbox for Granger causal connectivity analysis

    J. Neurosci. Meth.

    (2010)
  • W. Wei

    The effect of temporal aggregation on parameter estimation in distributed lag model

    J. Econometrics

    (1978)
  • A. Weiss

    Systematic sampling and temporal aggregation in time series models

    J. Econometrics

    (1984)
  • J. Zhang

    On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias

    Artif. Intell.

    (2008)
  • D. Zhou et al.

    MATLAB toolbox for functional connectivity

    Neuroimage

    (2009)
  • L. Baccalá et al.

    Partial directed coherence: a new concept in neural structure determination

    Biol. Cybern.

    (2001)
  • O. Banerjee et al.

    Convex optimization techniques for fitting sparse Gaussian graphical models

  • R. Buxton et al.

    Dynamics of blood flow and oxygenation changes during brain activation: the balloon model

    Magn. Reson. Med.

    (1998)
  • D. Chickering

    Optimal structure identification with greedy search

    J. Mach. Learn. Res.

    (2003)
  • David, O., in press. fMRI connectivity, meaning and empiricism: Comments on: Roebroeck et al. The identification of...
  • O. David et al.

    Identifying neural drivers with functional MRI: an electrophysiological validation

    PLoS Biol.

    (2008)
  • F. de Pasquale et al.

    Temporal dynamics of spontaneous MEG activity in brain networks

    Proc. Natl Acad. Sci.

    (2010)
  • Cited by (0)

    View full text