Abstract
The ability to make inferences about the current state of a dynamic process requires ongoing assessments of the stability and reliability of data generated by that process. We found that these assessments, as defined by a normative model, were reflected in nonluminance-mediated changes in pupil diameter of human subjects performing a predictive-inference task. Brief changes in pupil diameter reflected assessed instabilities in a process that generated noisy data. Baseline pupil diameter reflected the reliability with which recent data indicate the current state of the data-generating process and individual differences in expectations about the rate of instabilities. Together these pupil metrics predicted the influence of new data on subsequent inferences. Moreover, a task- and luminance-independent manipulation of pupil diameter predictably altered the influence of new data. Thus, pupil-linked arousal systems can help to regulate the influence of incoming data on existing beliefs in a dynamic environment.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Change history
17 January 2017
In the version of this article initially published, equation (7) was wrong. The error has been corrected in the HTML and PDF versions of the article.
References
Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).
Nassar, M.R., Wilson, R.C., Heasly, B. & Gold, J.I. An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J. Neurosci. 30, 12366–12378 (2010).
Yu, A.J. & Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005).
Nieuwenhuis, S., De Geus, E.J. & Aston-Jones, G. The anatomical and functional relationship between the P3 and autonomic components of the orienting response. Psychophysiology 2, 162–175 (2011).
Aston-Jones, G. & Cohen, J.D. An integrative theory of locus coeruleus–norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450 (2005).
Jepma, M. & Nieuwenhuis, S. Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory. J. Cogn. Neurosci. 23, 1587–1596 (2011).
Gilzenrat, M.S., Nieuwenhuis, S., Jepma, M. & Cohen, J.D. Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn. Affect. Behav. Neurosci. 10, 252–269 (2010).
Krugman, H.E. Some applications of pupil measurement. J. Mark. Res. 1, 15–19 (1964).
Granholm, E. & Steinhauer, S.R. Pupillometric measures of cognitive and emotional processes. Int. J. Psychophysiol. 52, 1–6 (2004).
Schmidt, H.S. & Fortin, L.D. Electronic pupillography in disorders of arousal. in Sleeping and Waking Disorders: Indication and Technique (ed. Guilleminault, C.) 127–143 (Addison-Wesley, 1982).
Kahneman, D. & Beatty, J. Pupil diameter and load on memory. Science 154, 1583–1585 (1966).
Richer, F. & Beatty, J. Contrasting effects of response uncertainty on the task-evoked pupillary response and reaction time. Psychophysiology 24, 258–262 (1987).
Hakerem, G., Sutton, S. & Zubin, J. Pupillary reactions to light in schizophrenic patients and normals. Ann. NY Acad. Sci. 105, 820–831 (1964).
Einhäuser, W., Stout, J., Koch, C. & Carter, O. Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proc. Natl. Acad. Sci. USA 105, 1704–1709 (2008).
Van Olst, E.H., Heemstra, M.L. & Ten Kortenaar, T. Stimulus significance and the orienting reaction. in The Orienting Reflex in Humans: International Conference Proceedings (eds. Kimmel, H., Olst, E.H. & van Orlebeke, J.F.) 521–547 (Erlbaum, 1979).
Sutton, R.S. & Barto, A.G. Reinforcement Learning: An Introduction (MIT Press, Cambridge, Massachusetts, 1998).
Adams, R.P. & MacKay, D.J.C. Bayesian online changepoint detection. Preprint at <http://arxiv.org/abs/0710.3742v1> (2007).
Fearnhead, P. & Liu, Z. On-line inference for multiple changepoint problems. J. R. Stat. Soc. Series B Stat. Methodol. 69, 589–605 (2007).
Wilson, R.C., Nassar, M.R. & Gold, J.I. Bayesian online learning of the hazard rate in change-point problems. Neural Comput. 22, 2452–2476 (2010).
Yerkes, R.M. & Dodson, J.D. The relation of strength of stimulus to rapidity of habit-formation. J. Comp. Neurol. Psychol. 18, 459–482 (1908).
Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).
Raisig, S., Welke, T., Hagendorf, H. & van der Meer, E. I spy with my little eye: detection of temporal violations in event sequences and the pupillary response. Int. J. Psychophysiol. 76, 1–8 (2010).
Friedman, D., Hakerem, G., Sutton, S. & Fleiss, J.L. Effect of stimulus uncertainty on the pupillary dilation response and the vertex evoked potential. Electroencephalogr. Clin. Neurophysiol. 34, 475–484 (1973).
Preuschoff, K., 't Hart, B.M. & Einhäuser, W. Pupil dilation signals surprise: evidence for noradrenaline's role in decision making. Front. Neurosci. 5, 115 (2011).
Aston-Jones, G., Ennis, M., Pieribone, V.A., Nickell, W.T. & Shipley, M.T. The brain nucleus locus coeruleus: restricted afferent control of a broad efferent network. Science 234, 734–737 (1986).
Sara, S.J., Vankov, A. & Hervé, A. Locus coeruleus-evoked responses in behaving rats: a clue to the role of noradrenaline in memory. Brain Res. Bull. 35, 457–465 (1994).
Aston-Jones, G., Rajkowski, J. & Kubiak, P. Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience 80, 697–715 (1997).
Tully, K. & Bolshakov, V.Y. Emotional enhancement of memory: how norepinephrine enables synaptic plasticity. Mol. Brain 3, 15 (2010).
Harley, C.W. A role for norepinephrine in arousal, emotion and learning? Limbic modulation by norepinephrine and the Kety hypothesis. Prog. Neuropsychopharmacol. Biol. Psychiatry 11, 419–458 (1987).
Corbetta, M., Patel, G. & Shulman, G.L. The reorienting system of the human brain: from environment to theory of mind. Neuron 58, 306–324 (2008).
Bouret, S. & Sara, S.J. Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 28, 574–582 (2005).
Critchley, H.D., Mathias, C.J. & Dolan, R.J. Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron 29, 537–545 (2001).
Critchley, H.D. Neural mechanisms of autonomic, affective, and cognitive integration. J. Comp. Neurol. 493, 154–166 (2005).
Krugel, L.K., Biele, G., Mohr, P.N., Li, S.C. & Heekeren, H.R. Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proc. Natl. Acad. Sci. USA 106, 17951–17956 (2009).
Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647–656 (2007).
Acknowledgements
We thank J. Cohen, S. du Lac, L. Ding, Y. Li and J. Nassar for helpful comments. This work was supported by the US National Institutes of Health (R01 EY015260, F31 MH093099 and T90 DA22763), the McKnight Endowment Fund for Neuroscience, the Burroughs-Wellcome Fund and the Sloan Foundation.
Author information
Authors and Affiliations
Contributions
M.R.N., J.I.G. and B.H. designed the experiment and tasks. M.R.N., K.M.R. and K.P. collected and analyzed data. M.R.N. and R.C.W. developed and applied the reduced Bayesian model. M.R.N. and J.I.G. wrote the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Rights and permissions
About this article
Cite this article
Nassar, M., Rumsey, K., Wilson, R. et al. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15, 1040–1046 (2012). https://doi.org/10.1038/nn.3130
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.3130
This article is cited by
-
Specifying the timescale of early life unpredictability helps explain the development of internalising and externalising behaviours
Scientific Reports (2024)
-
Out of control: computational dynamic control dysfunction in stress- and anxiety-related disorders
Discover Mental Health (2024)
-
Do dogs preferentially encode the identity of the target object or the location of others’ actions?
Animal Cognition (2024)
-
Adaptive learning from outcome contingencies in eating-disorder risk groups
Translational Psychiatry (2023)
-
Trait anxiety is associated with hidden state inference during aversive reversal learning
Nature Communications (2023)