Network modelling methods for FMRI
Research Highlights
► We evaluate many methods for estimating brain networks from FMRI data ► Covariance-based methods are the most successful at finding network edges ► Finding network directionality is harder; Patel's tau performs the best ► Lag-based methods (e.g. Granger) perform badly ► The use of incorrect ROIs (e.g. from structural atlases) is dangerous
Introduction
Neuroimaging is used to study many aspects of the brain's function and structure; one area of rapidly increasing interest is the mapping of functional networks. Such mapping typically starts by identifying a set of functional “nodes”, and then attempts to estimate the set of connections or “edges” between these nodes. In some cases, the directionality of these connections is estimated, in an attempt to show how information flows through the network.
There are many ways to define network nodes. In the case of electrophysiological data, the simplest approach is to either consider each recorded channel as a node, or instead use spatial sources after source reconstruction (spatial localisation) has been carried out. In the case of FMRI, nodes are often defined as spatial regions of interest (ROIs), for example, as obtained from brain atlases or from functional localiser tasks. Alternatively, independent component analysis (ICA) can be run to define independent components (spatial maps and associated timecourses), which can be considered network nodes, although the extent to which this makes sense depends on the number of components extracted (the ICA dimensionality). If a low number of components is estimated (Kiviniemi et al., 2003), then it makes more sense to think of each component itself as a network. This will often include several non-contiguous regions, all having the same timecourse according to the ICA model, and hence within-component network analysis is not possible without further processing, such as splitting the components and re-estimating each resulting node's timeseries. Furthermore, between-component network analysis is quite possibly not reasonable, as each component will in itself constitute a gross, complex functional system. However, if a higher number of components is estimated (Kiviniemi et al., 2009), these are more likely to be smaller, isolated regions (functional parcels), which can more sensibly be then considered as nodes for use in network analysis.
Once the nodes are defined, each has its own associated timecourse. These are then used to estimate the connections between nodes—in general, the more similar the timecourses are between any given pair of nodes, the more likely it is that there is a functional connection between those nodes. Of course, correlation (between two timeseries) does not necessarily imply either causality (in itself it tells one nothing about the direction of information flow), or whether the functional connection between two nodes is direct (there may be a third node “in-between” the two under consideration, or a third node may be feeding into the two, without a direct connection existing between them). This distinction between apparent correlation and true, direct functional connection (sometimes referred to as the distinction between functional and effective connectivity respectively; Friston, 1994) is very important if one cares about correctly estimating the network. For example, in a 3-node network where A→ B →C, and with external inputs (or at least added noise that feeds around the network) for all nodes, then all three nodes' timeseries will be correlated with each other, so the “network estimation method” of simple correlation will incorrectly estimate a triangular network. However, another simple estimation method, partial correlation, can correctly estimate the true network; this works by taking each pair of timeseries in turn, and regressing out the third from each of the two timeseries in question, before estimating the correlation between the two. If B is regressed out of A and C, there will no longer be any correlation between A and C, and hence the spurious third edge of the network (A–C) is correctly eliminated.
The question of directionality is also often of interest, but in general is harder to estimate than whether a connection exists or not. For example, many methods, such as the two mentioned above (full correlation and partial correlation) give no directional information at all. The methods that do attempt to estimate directionality fall into three general classes. The first class is “lag-based”, the most common example being Granger causality (Granger, 1969). Here it is assumed that if one timeseries looks like a time-shifted version of the other, then the one with temporal precedence caused the other, giving an estimation of connection directionality. The second class is based on the idea of conditional independence, and generally starts by estimating the (zero-lag) covariance matrix between all nodes' timeseries (hence such methods are based on the same raw measure of connectivity as correlation-based approaches—but attempt to go further in utilising this matrix to draw more complex inferences about the network). Such methods may look at the probability of pairs of variables conditional on sets of other variables; for example, Bayes net methods (Ramsey et al., 2010) in general estimate directionality by first orienting “unshielded colliders” (paths of the form A→ B ←C) and then drawing inferences based on algorithm-specific assumptions regarding what further orientations are implied by these colliders. The third class of methods utilises higher order statistics than just the covariance; for example, Patel's pairwise conditional probability approach (Patel et al., 2006) looks at the probability of A given B, and B given A, with asymmetry in these probabilities being interpreted as indicating causality.
A large number of network estimation methods have been used in the neuroimaging literature, with varying degrees of validation. The closer a given modality's data is to the underlying neural sources, the simpler it is to interpret the data and analyses resulting from it. In the case of FMRI, the data is a relatively indirect measure of the neural activity, being distanced from the underlying sources by many confounding stages, particularly the nonlinear neuro-vascular coupling that adds (generally unknown amounts of) significant blurring and delay to the neural signal (Buxton et al., 1998). This means that very careful validation is necessary before network estimation methods applied to FMRI data can be safely interpreted, and, unfortunately, it is too often the case that careful, sufficiently rich, validation is not carried out before real data is analysed and interpreted. Several approaches have been applied to electrophysiological data, and have been well validated for that application domain; however, because FMRI data is so much further removed from the underlying sources of interest than is generally the case with the various electrophysiological modalities, FMRI-specific validations are of particular importance. We concentrate solely on FMRI data in this paper. We simulate resting FMRI data, although the results will also in general be relevant for task FMRI (in fact, the input timings generated in the simulations could equally be viewed as simulating an event-related task FMRI experiment).
The purpose of this work is to apply a rich biophysical FMRI model to a range of network scenarios, in order to provide a thorough simulation-based evaluation of many different network estimation methods. We have compared their relative sensitivities to finding the presence of a direct network connection, their ability to correctly estimate the direction of the connection, and their robustness against various problems that can arise in real data. We find that some of the methods in common use are not effective approaches, and even can easily give erroneous results.
Section snippets
Methods: Simulations
Networks of varied complexity were used to simulate rich, realistic BOLD timeseries. The simulations were based upon the dynamic causal modelling (DCM; Friston et al., 2003) FMRI forward model, which uses the nonlinear balloon model (Buxton et al., 1998) for the vascular dynamics, sitting on top of a neural network model. We now describe in detail how our simulations were generated. Specific simulation parameters given are true in general for most of the evaluations, except where particular
Methods: Network modelling methods tested
We now give a brief description of each of the methods tested. Where minor variants of each main method (including alternative choices in controlling parameters) performed universally worse than other variants, we exclude the unsuccessful variants from further consideration in the paper, in order to maximise the clarity of presentation. We describe all variants tested (including descriptions of those that were rejected) within this section.
Results
We begin by explaining how we summarised the outputs from testing the different network modelling approaches. (For a summary of the specifications for all 28 simulations see Table 1.) Results from one of the most “typical” network scenarios (Sim2) are shown in Fig. 3, which has 10 nodes, 10 min FMRI sessions for each subject, TR = 3 s, measurement noise (thermal noise added onto the BOLD signal) of 1%, and HRF lag variability of ± 0.5 s.
For some of these plots we use the raw connection strength
Discussion
Although some of the different data scenarios generated quite variable sets of results, a general picture does emerge, that should be applicable across a large fraction of real FMRI experiments.
With respect to estimating the presence of a network connection, the overall results suggest that the “Top-3” (Partial correlation, ICOV and the Bayes net methods) often perform excellently, with a sensitivity of more than 90% on “typical” data. The Bayes net methods are as good as, or slightly better
Acknowledgments
We are very grateful to: Gopikrishna Deshpande for helpful discussions, Aslak Grinsted for providing the Crosswavelet and Wavelet Coherence toolbox, Chandler Lutz for providing the pairwise Granger causality code, Alard Roebroeck for helpful discussions, Alois Schlögl for providing the BioSig toolbox, Mark Schmidt for providing the regularised inverse covariance code (and for helpful discussions), Anil Seth for providing the Causal Connectivity Analysis toolbox (and for helpful discussions),
References (59)
- et al.
Time-frequency dynamics of resting-state brain connectivity measured with fMRI
Neuroimage
(2010) - et al.
Mapping and correction of vascular hemodynamic latency in the BOLD signal
Neuroimage
(2008) - et al.
Variational Bayesian identification and prediction of stochastic nonlinear dynamic causal models
Phys. D Nonlinear Phenom.
(2009) - et al.
A comparative study of synchrony measures for the early diagnosis of Alzheimer's disease based on EEG
Neuroimage
(2010) - et al.
Effect of hemodynamic variability on Granger causality analysis of fMRI
Neuroimage
(2010) - et al.
Dynamic causal modelling
Neuroimage
(2003) - et al.
Investigating directed cortical interactions in time-resolved fMRI data using vector autoregressive modeling and Granger causality mapping
Magn. Reson. Imaging
(2003) - et al.
Partial Granger causality—eliminating exogenous inputs and latent variables
J. Neurosci. Meth.
(2008) - et al.
Variation of BOLD hemodynamic responses across subjects and brain regions and their effects on statistical analyses
Neuroimage
(2004) - et al.
Dynamic Granger causality based on Kalman filter for evaluation of functional network connectivity in fMRI data
NeuroImage
(2010)