Thumbnail
Access Restriction
Subscribed

Author Rivest, Francois ♦ Kalaska, John F. ♦ Bengio, Yoshua
Source SpringerLink
Content type Text
Publisher Springer Berlin Heidelberg
File Format PDF
Copyright Year ©2013
Language English
Subject Domain (in DDC) Technology ♦ Medicine & health
Subject Keyword Time representation learning ♦ Temporal-difference learning ♦ Long short-term memory networks ♦ Dopamine ♦ Conditioning ♦ Reinforcement learning ♦ Neurosciences ♦ Computer Application in Life Sciences ♦ Neurobiology ♦ Bioinformatics ♦ Statistical Physics, Dynamical Systems and Complexity
Abstract Dopaminergic models based on the temporal-difference learning algorithm usually do not differentiate trace from delay conditioning. Instead, they use a fixed temporal representation of elapsed time since conditioned stimulus onset. Recently, a new model was proposed in which timing is learned within a long short-term memory (LSTM) artificial neural network representing the cerebral cortex (Rivest et al. in J Comput Neurosci 28(1):107–130, 2010). In this paper, that model’s ability to reproduce and explain relevant data, as well as its ability to make interesting new predictions, are evaluated. The model reveals a strikingly different temporal representation between trace and delay conditioning since trace conditioning requires working memory to remember the past conditioned stimulus while delay conditioning does not. On the other hand, the model predicts no important difference in DA responses between those two conditions when trained on one conditioning paradigm and tested on the other. The model predicts that in trace conditioning, animal timing starts with the conditioned stimulus offset as opposed to its onset. In classical conditioning, it predicts that if the conditioned stimulus does not disappear after the reward, the animal may expect a second reward. Finally, the last simulation reveals that the buildup of activity of some units in the networks can adapt to new delays by adjusting their rate of integration. Most importantly, the paper shows that it is possible, with the proposed architecture, to acquire discharge patterns similar to those observed in dopaminergic neurons and in the cerebral cortex on those tasks simply by minimizing a predictive cost function.
ISSN 03401200
Age Range 18 to 22 years ♦ above 22 year
Educational Use Research
Education Level UG and PG
Learning Resource Type Article
Publisher Date 2013-11-21
Publisher Place Berlin/Heidelberg
e-ISSN 14320770
Journal Biological Cybernetics
Volume Number 108
Issue Number 1
Page Count 26
Starting Page 23
Ending Page 48


Open content in new tab

   Open content in new tab
Source: SpringerLink