Review
Bayesian modeling of flexible cognitive control

https://doi.org/10.1016/j.neubiorev.2014.06.001Get rights and content

Highlights

  • We review Bayesian graphical models of cognitive control processes.

  • We highlight lack of mechanisms for time-varying adjustment of control.

  • We present a Bayesian control model with volatility-driven learning.

  • This model provides flexible, context-sensitive prediction of control demand.

  • Bayesian modeling of cognitive control is a promising new research avenue.

Abstract

“Cognitive control” describes endogenous guidance of behavior in situations where routine stimulus-response associations are suboptimal for achieving a desired goal. The computational and neural mechanisms underlying this capacity remain poorly understood. We examine recent advances stemming from the application of a Bayesian learner perspective that provides optimal prediction for control processes. In reviewing the application of Bayesian models to cognitive control, we note that an important limitation in current models is a lack of a plausible mechanism for the flexible adjustment of control over conflict levels changing at varying temporal scales. We then show that flexible cognitive control can be achieved by a Bayesian model with a volatility-driven learning mechanism that modulates dynamically the relative dependence on recent and remote experiences in its prediction of future control demand. We conclude that the emergent Bayesian perspective on computational mechanisms of cognitive control holds considerable promise, especially if future studies can identify neural substrates of the variables encoded by these models, and determine the nature (Bayesian or otherwise) of their neural implementation.

Section snippets

Cognitive control as statistical prediction

“Cognitive control” describes the ability to guide one's behavior in line with internal goals. A key characteristic of cognitive control is thought to be flexibility: control processes must be capable of dynamically adapting (both qualitatively and quantitatively) to ongoing changes in the environment. How this type of contextual regulation of control occurs (in the absence of an all-knowing homunculus) is a key question in current cognitive psychology and neuroscience research. In the present

Overview of Bayesian methods

Bayes’ theorem can be written as follows:P(Y|X)=P(X|Y)P(Y)P(X)where X, Y are random variables (e.g., sensory input, internal states, motor output, etc.). Unlike conventional variables, the value of a random variable can vary due to randomness. Thus, a random variable is often represented in the probability distribution of its possible values. This equation means that the conditional probability of Y given X could be calculated using the probabilities of X, Y, and the conditional probability of X

A Bayesian model of flexible conflict-control

Here, we propose a Bayesian model that can account for the flexibility of cognitive control over conflict in a non-stationary environment. The modeling done relies on the ability to perform statistical inference, taking the perspective that the regulation of cognitive control should be considered as a process of predicting the optimal amount of cognitive control required in a given context. To achieve this contextual flexibility, the model estimates future conflict from previous experience and,

How “Bayesian” is cognitive control?

One important avenue for future work in this context is to evaluate to what extent (or in what sense) cognitive control might actually be Bayesian in nature. According to Bowers and Davis (2012), there are three levels at which one can use Bayesian methods in modeling cognitive processes: as computational tools, for generating “optimal” benchmarks for cognitive processes, and for modeling the actual neural computations carried out by the brain. The Bayesian models reviewed in this paper, along

Conclusion

The flexibility of cognitive control enables the brain to adaptively adjust the degree of top-down biasing in a dynamically changing environment. This adjustment can be cast as a prediction of control demand, which can be optimally achieved via Bayesian belief propagation. Using Bayesian models, previous studies have successfully modeled various phenomena of cognitive control. Yet, those models usually depend on specific (and post hoc) selection of parameters to achieve optimal performance.

Acknowledgments

We thank Tim Behrens for sharing code and Chris Summerfield for discussion of the model. This work was funded in part by National Institute of Mental Health (NIMH) grants R01MH087610 (T.E.) and R01MH097965 (T.E.).

References (97)

  • D. Servan-Schreiber et al.

    Dopamine and the mechanisms of cognition: Part I. A neural network model predicting dopamine effects on selective attention

    Biol. Psychiatry

    (1998)
  • M. Silvetti et al.

    Value and prediction error estimation account for volatility effects in ACC: a model-based fMRI study

    Cortex: J. Devoted Study Nerv. Syst. Behav.

    (2013)
  • C. Summerfield et al.

    Perceptual classification in a rapidly changing environment

    Neuron

    (2011)
  • J.B. Tenenbaum et al.

    Theory-based Bayesian models of inductive learning and reasoning

    Trends Cogn. Sci.

    (2006)
  • M. Torres-Quesada et al.

    Dissociating proportion congruent and conflict adaptation effects in a Simon–Stroop procedure

    Acta Psychol. (Amst.)

    (2013)
  • T. Verguts et al.

    Adaptation by binding: a learning account of cognitive control

    Trends Cogn. Sci.

    (2009)
  • K.G. Volz et al.

    Predicting events of varying probability: uncertainty investigated by fMRI

    Neuroimage

    (2003)
  • K.G. Volz et al.

    Why am I unsure? Internal and external attributions of uncertainty dissociated by fMRI

    Neuroimage

    (2004)
  • C.N. White et al.

    Diffusion models of the flanker task: discrete versus gradual attentional selection

    Cogn. Psychol.

    (2011)
  • M. Wittfoth et al.

    Comparison of two Simon tasks: neuronal correlates of conflict resolution based on coherent motion perception

    Neuroimage

    (2006)
  • A.J. Yu et al.

    Uncertainty, neuromodulation, and attention

    Neuron

    (2005)
  • W.H. Alexander et al.

    Medial prefrontal cortex as an action-outcome predictor

    Nat. Neurosci.

    (2011)
  • C. Amiez et al.

    Reward encoding in the monkey anterior cingulate cortex

    Cereb. Cortex

    (2006)
  • D.R. Bach et al.

    Knowing how much you don’t know: a neural organization of uncertainty estimates

    Nat. Rev. Neurosci.

    (2012)
  • D.M. Barch et al.

    Anterior cingulate cortex and response conflict: effects of response modality and processing domain

    Cereb. Cortex

    (2001)
  • T.E. Behrens et al.

    Learning the value of information in an uncertain world

    Nat. Neurosci.

    (2007)
  • C. Blais et al.

    Item-specific adaptation and the conflict-monitoring hypothesis: a computational model

    Psychol. Rev.

    (2007)
  • M. Botvinick et al.

    Conflict monitoring versus selection-for-action in anterior cingulate cortex

    Nature

    (1999)
  • M.M. Botvinick et al.

    Conflict monitoring and cognitive control

    Psychol. Rev.

    (2001)
  • J.S. Bowers et al.

    Bayesian just-so stories in psychology and neuroscience

    Psychol. Bull.

    (2012)
  • T.S. Braver et al.

    Explaining the may varieties of working memory variation: dual mechanisms of cognitive control

  • J.M. Bugg

    Dissociating levels of cognitive control: the case of Stroop interference

    Psychol. Sci.

    (2012)
  • J.M. Bugg et al.

    List-wide control is not entirely elusive: evidence from picture-word Stroop

    Psychon. Bull. Rev.

    (2011)
  • J.M. Bugg et al.

    In support of a distinction between voluntary and stimulus-driven control: a review of the literature on proportion congruent effects

    Front. Psychol.

    (2012)
  • J.M. Bugg et al.

    Converging evidence for control of color-word Stroop interference at the item level

    J. Exp. Psychol. Hum. Percept. Perform.

    (2013)
  • C.S. Carter et al.

    Anterior cingulate cortex, error detection, and the online monitoring of performance

    Science

    (1998)
  • J.D. Cohen et al.

    On the control of automatic processes: a parallel distributed processing account of the Stroop effect

    Psychol. Rev.

    (1990)
  • B. De Martino et al.

    Confidence in value-based choice

    Nat. Neurosci.

    (2013)
  • H.E. den Ouden et al.

    Striatal prediction error modulates cortical coupling

    J. Neurosci.: Off. J. Soc. Neurosci.

    (2010)
  • R. Desimone et al.

    Neural mechanisms of selective visual attention

    Annu. Rev. Neurosci.

    (1995)
  • J. Duncan

    An adaptive coding model of neural function in prefrontal cortex

    Nat. Rev. Neurosci.

    (2001)
  • T. Egner

    Congruency sequence effects and cognitive control

    Cogn. Affect. Behav. Neurosci.

    (2007)
  • T. Egner et al.

    Going, going, gone: characterizing the time-course of congruency sequence effects

    Front. Psychol.

    (2010)
  • T. Egner et al.

    Dissociable neural systems resolve conflict from emotional versus nonemotional distracters

    Cereb. Cortex

    (2008)
  • T. Egner et al.

    Cognitive control mechanisms resolve conflict through cortical amplification of task-relevant information

    Nat. Neurosci.

    (2005)
  • B.A. Eriksen et al.

    Effects of noise letters upon identification of a target letter in a non- search task

    Percept Psychophys

    (1974)
  • C.D. Fiorillo et al.

    Discrete coding of reward probability and uncertainty by dopamine neurons

    Science

    (2003)
  • C.D. Fiorillo et al.

    Evidence that the delay-period activity of dopamine neurons corresponds to reward uncertainty rather than backpropagating TD errors

    Behav. Brain Funct.: BBF

    (2005)
  • Cited by (0)

    View full text