Original contribution
Neural dynamics of adaptive timing and temporal discrimination during associative learning

https://doi.org/10.1016/0893-6080(89)90026-9Get rights and content

Abstract

A neural network model that controls behavioral timing is described and simulated. This model, called the Spectral Timing Model, controls a type of timing whereby an animal or robot can learn to wait for an expected goal by discounting expected nonoccurrences of a goal object until the expected time of arrival of the goal. If the goal object does not then materialize, the animal can respond to unexpected nonoccurrences of the goal with appropriate changes in information processing and exploratory behavior. The model is a variant of the gated dipole model of opponent processing. When the gated dipole model is generalized to include a spectrum of cellular response rates within a large population of cells, the model's total output signal generates accurate learned timing properties that collectively provide a good quantitative fit to animal learning data. In particular, the Spectral Timing Model utilizes the habituative transmitter gates and adaptive long-term memory traces that are characteristic of gated dipole models. The Spectral Timing Model is embedded into an Adaptive Resonance Theory (ART) neural architecture for the learning of correlations between internal representations of recognition codes and reinforcement codes. This type of learning is called conditioned reinforcer learning. The two types of internal representations are called sensory representations (S) and drive representations (D). Activation of a drive representation D by the Spectral Timing Model inhibits output signals from the orienting subsystem (A) of the ART architecture and activates a motor response. The inhibitory pathway helps to prevent spurious resets of short-term memory, forgetting, and orienting responses from being caused by events other than the goal object prior to the expected arrival time of the goal. Simulated data properties include the inverted U in learning as a function of the interstimulus interval (ISI) that occurs between onset of the conditioned stimulus (CS) and the unconditioned stimulus (US); correlations of peak time, standard deviation, Weber fraction, and peak amplitude of the conditioned response as a function of the ISI; increase of conditioned response amplitude, but not its timing, with US intensity; speed-up of the timing circuit by an increase in CS intensity or by drugs that increase concentrations of brain dopamine or acetylcholine; multiple timing peaks in response to learning conditions using multiple ISIs; and conditioned timing of cell activation within the hippocampus and of the contingent negative variation (CNV) event-related potential. The results on speed-up by drugs that increase brain concentrations of dopamine and acetylcholine support a 1972 prediction that the gated dipole habituative transmitter is a catecholamine and its long-term memory trace transmitter is acetylcholine. It is noted that the timing circuit described herein is only one of several functionally distinct neural circuits for governing different types of timed behavior competence.

References (64)

  • D.M. Parker et al.

    Latency changes in the human visual evoked response to sinusoidal gratings

    Vision Research

    (1977)
  • D.M. Parker et al.

    The early waves of the visual evoked potential to sinusoidal gratings: Responses to quadrant stimulation as a function of spatial frequency

    Electroencephalography and Clinical Neurophysiology

    (1982)
  • G.T. Plant et al.

    Transient visually evoked potentials to the pattern reversal and onset of sinusoidal gratings

    Electroencephalography and Clinical Neurophysiology

    (1983)
  • W. Skrandies

    Scalp potential fields evoked by grating stimuli: Effects of spatial frequency and orientation

    Electroencephalography and Clinical Neurophysiology

    (1984)
  • A. Vassilev et al.

    Spatial frequency and pattern onset-offset response

    Vision Research

    (1983)
  • R.W. Black et al.

    Heart rate conditioning as a function of interstimulus interval in rats

    Psychonomic Science

    (1967)
  • R. Boice et al.

    The conditioned licking response in rats as a function of the CS-US interval

    Psychonomic Science

    (1965)
  • P.E. Burkhardt et al.

    CS and US duration effects in one-trial simultaneous fear conditioning as assessed by conditioned suppression of licking rats

    Animal Learning and Behavior

    (1978)
  • B.R. Cant et al.

    The effect of motivation on the contingent negative variation (CNV)

    Electroencephalography and Clinical Neurophysiology

    (1967)
  • G.A. Carpenter et al.

    ART 2: Stable self-organization of pattern recognition codes for analog input patterns

    Applied Optics

    (1987)
  • G.A. Carpenter et al.

    The ART of adaptive pattern recognition by a self-organizing neural network

    Computer

    (1988)
  • M.A. Cohen et al.

    Neural dynamics of speech and language coding: Developmental programs, perceptual grouping, and competition for short term memory

    Human Neurobiology

    (1986)
  • M.A. Cohen et al.

    Masking fields: A massively parallel architecture for learning, recognizing, and predicting multiple groupings of patterned data

    Applied Optics

    (1987)
  • J. Delacour et al.

    Conditioning to time: Evidence for a role of hippocampus from unit recording

    Neuroscience

    (1980)
  • I. Gormenzano et al.

    Twenty years of classical conditioning research with the rabbit

    Progress in Psychobiology and Physiological Psychology

    (1983)
  • S. Grossberg

    Adaptive pattern classification and universal recoding, II: Feedback, expectation, olfaction, and illusions

    Biological Cybernetics

    (1976)
  • S. Grossberg

    Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control

    (1982)
  • S. Grossberg et al.

    Neural dynamics of adaptive sensory-motor control: Ballistic eye movements

    (1986)
  • S. Grossberg et al.

    Neural dynamics of attentionally-modulated Pavlovian conditioning: Blocking, inter-stimulus interval, and secondary reinforcement

    Applied Optics

    (1987)
  • S. Grossberg et al.

    Neural dynamics of attentionally-modulated Pavlovian conditioning: Conditioned reinforcement, inhibition, and opponent processing

    Psychobiology

    (1987)
  • Cited by (270)

    • How associations become behavior

      2023, Neurobiology of Learning and Memory
    • A simple three layer excitatory-inhibitory neuronal network for temporal decision-making

      2020, Behavioural Brain Research
      Citation Excerpt :

      All normalized choice functions overlap for all parameter values when they are positioned on the normalized axis. A number of successful neural models of interval timing ability have been developed to account for the core features of timing behavior [14,15,26,20,21]. However, these models have not been extended to explain temporal decision-making restricting the scope of their applicability [but see [6]].

    View all citing articles on Scopus

    Supported in part by the Air Force Office of Scientific Research (AFOSR F49620-86-C-0037 and AFOSR F49620-87-C-0018) and the National Science Foundation (NSF IRI-84-17756).

    View full text