A Review of Computational Classical Conditioning Models
Classical conditioning (CC), which is a basic learning phenomenon, used to explain basic emotions, such as fear, phobia, and reflexes. It was introduced by Pavlov in 1927, and since then it has been investigated in psychological, behavioral, memory, neuroscience, and neurobiology perspectives. It has been used for investigating consumer behavior, conditional fear acquisition and extinction, response extinction and robotic control. Additionally, a large number of methods were proposed to model the CC learning stage. Unfortunately, none of them could model all outcomes of CC. In this study, these computational models, their usages and also the papers about their comparisons are reviewed. It obvious from this review that there is a high requirement to a model, which has a capability to model each feature of CC.
Classical Conditioning, Pavlov, Computational Model, Behavioral Learning
[1]
Pavlov, Ivan Petrovich. Conditioned reflexes. DoverPublications. com, 1927.
[2]
Martin Inderbitzin, Ivan Herreros-Alonso and Paul F.M.J. Verschure, “An Integrated Computational Model of the Two Phase Theory of Classical Conditioning”, IEEE, 2010
[3]
Dugatkin, LA., “Principles of animal behavior” New York, New York, USA: WW Norton, 2004.
[4]
Raja L. Clouse, Soowon Kim, and Manjula B. Waldron, “An Adaptive Threshold Learning Algorithm for Classical Conditioning”, Proceedings - 19th International Conference - IEEE/EMBS Oct. 30 - Nov. 2, 1997 Chicago, IL. USA, pg. 1380-1382.
[5]
Salotti, Jean Marc, and Florent Lepretre. "Classical and Operant Conditioning as Roots of Interaction for Robots." Proceedings of the Workshop from Motor to Interaction Learning in Robots, Conference on Intelligent Robotics Systems, Nice. 2008.
[6]
Malaka, Rainer, and Martin Hammer. "Real-time models of classical conditioning." Neural Networks, 1996, IEEE International Conference on. Vol. 2. IEEE, 1996.
[7]
Louwrence D Erasmus, “Classical Conditioning Implementation with RealNeuron Networks”, IEEE, 2007.
[8]
Gonzalo Miguez, Mario A. Laborda and Ralph R. Miller, “Classical conditioning and pain: Conditioned analgesia and hyperalgesia”, Acta Psychologica 145 (2014) 10–20.
[9]
Derick H. Lindquist, Greta Sokoloff, Joseph E. Steinmetz, “Ethanol-exposed neonatal rats are impaired as adults in classical eyeblink conditioning at multiple unconditioned stimulus intensities”, Brain Research 1150 (2007) 155 – 166.
[10]
Christina Dalla, Tracey J. Shors, “Sex differences in learning processes of classical and operant conditioning”, Physiology & Behavior 97 (2009) 229–238.
[11]
Mirte Brom, Stephanie Both, Ellen Laan,Walter Everaerd, Philip Spinhoven, “The role of conditioning, learning and dopamine in sexual behavior: A narrative review of animal and human studies”, Neuroscience and Biobehavioral Reviews 38 (2014) 38–59.
[12]
Ole Åsli, Magne Arve Flaten, “Conditioned facilitation of the unconditioned reflex after classical eyeblink conditioning”, International Journal of Psychophysiology 67 (2008) 17–22.
[13]
Tomer Shechner, Melanie Hong, Jennifer C. Britton, Daniel S. Pine, Nathan A. Fox, “Fear conditioning and extinction across development: Evidence from human studies and animal models”, Biological Psychology 100 (2014) 1–12.
[14]
Bram Vervliet, Maarten Geens, “Fear generalization in humans: Impact of feature learning on conditioning and extinction”, Neurobiology of Learning and Memory 113 (2014) 143–148.
[15]
Masayuki Yoshida, Izumi Okamura, Kazumasa Uematsu, “Involvement of the cerebellum in classical fear conditioning in goldfish”, Behavioural Brain Research, 153 (2004) 143–148.
[16]
Kazuyuki Yamada, “Strain differences of selective attention in mice: Effect of Kamin blocking on classical fear conditioning”, Behavioural Brain Research 213 (2010) 126–129.
[17]
Thomas M. Ricarta, Xilu Jiao, Kevin C.H. Pang, Kevin D. Beck, Richard. J. Servatius, “Classical and instrumental conditioning of eyeblink responses inWistar–Kyoto and Sprague–Dawley rats”, Behavioural Brain Research 216 (2011) 414–418.
[18]
Miriam S. Nokia, Markku Penttonen, Tapani Korhonen, Jan Wikgren, “Hippocampal theta (3–8 Hz) activity during classical eyeblink conditioning in rabbits”, Neurobiology of Learning and Memory 90 (2008) 62–70.
[19]
Elisabetta Baldi, Corrado Bucherelli, “Entorhinal cortex contribution to contextual fear conditioning extinction and reconsolidation in rats”, Neurobiology of Learning and Memory 110 (2014) 64–71.
[20]
Andrew R. Delamater, R. Frederick Westbrook, “Psychological and neural mechanisms of experimental extinction: A selective review”, Neurobiology of Learning and Memory 108 (2014) 38–51.
[21]
Travis P. Todd, Drina Vurbic, Mark E. Bouton, “Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning”, Neurobiology of Learning and Memory 108 (2014) 52–64.
[22]
M.E. Bouton, E.W. Moody, “Memory processes in classical conditioning”, Neuroscience and Biobehavioral Reviews 28 (2004) 663–674.
[23]
Luis Aguado, “Neuroscience of Pavlovian Conditioning: A Brief Review”, The Spanish Journal of Psychology , 2003, Vol. 6, No. 2, 155-167.
[24]
Jonathan D. Raybuck, K. Matthew Lattal, “Bridging the interval: Theory and neurobiology of trace conditioning”, Behavioural Processes 101 (2014) 103–111.
[25]
R. F. Thompson and J. E. Steinmetz, “The Role of the Cerebellum in Classical Conditioning of Discrete Behavioral Responses”, Neuroscience 162 (2009) 732–755.
[26]
Germund Hesslow, Dan-Anders Jirenhed, Anders Rasmussen, Fredrik Johansson, “Classical conditioning of motor responses:What is the learning mechanism?”, Neural Networks 47 (2013) 81–87.
[27]
Stephen D. Berry, Loren C. Hoffmann, “Hippocampal theta-dependent eyeblink classical conditioning: Coordination of a distributed learning system”, Neurobiology of Learning and Memory 95 (2011) 185–189.
[28]
Drew C. Thompson, Irving Z. Basanez, Douglas A. Baxter and John H. Byrne, “A Computational Study of Operant and Classical Conditioning in a Central Pattern Generator Neural Circuit”.
[29]
Blackburn, Michael R. "A general architecture for classical conditioning of perceptual-motor sequences." Neural Networks, 1990., 1990 IJCNN International Joint Conference on. IEEE, 1990.
[30]
Ahmed A. Moustafa, Catherine E. Myers, Mark A. Gluck, “A neurocomputational model of classical conditioning phenomena: A putative role for the hippocampal region in associative learning”, Brain Research 1276 (2009) 180 – 195.
[31]
Torsten Lehmann, “Classical Conditioning with Pulsed Integrated Neural Networks: Circuits and System”, IEEE Transactions on Circuits and Systems—II: Analog and Digital Signal Processing, 45(6), (1998), 720-728.
[32]
Lehmann, Torsten. "Eccopunch: the Edinburgh classical conditioning pulsed neural chip." Circuits and Systems, 1997. ISCAS'97., Proceedings of 1997 IEEE International Symposium on, 1997.
[33]
Magri, S., Pepe, I., Traverso, S., & Nicolini, C. (1993, October). A model of associative learning and classical conditioning in Aplysia californica: a preliminary study. In Neural Networks, 1993. IJCNN'93-Nagoya. Proceedings of 1993 International Joint Conference on Vol. 3, pp. 2694-2697, IEEE.
[34]
B. Yeşilyaprak, B. Aydın, M. Bilgin, G. Can, K. Ersanlı, M. Kılıç, İ. Kısaç, İ. Korkmaz, H. Küçükkaragöz, Ş. Küllahoğlu, B. Öztürk ve E. Uçar, Eğitim Psikolojisi Gelişim-Öğrenme-Öğretim, Ankara: Pegem Akademi, 2012.
[35]
R. L. Veznedaroğlu ve A. O. Özgür, “Öğrenme Stilleri: Tanımlamalar, Modeller ve İşlevleri”, İlköğretim Online, 4(2), (2005), pp. 1-16.
[36]
Olson, M. H., & Hergenhahn, B. R. (2009). An introduction to theories of learning. Pearson/Prentice Hall.
[37]
Garcia, J., Koelling, R. A. (1966). Relation of cue to consequences in avoidance learning. Psychonomic Science, 4, 123-124.
[38]
Kamin, L. J. (1968). Attention–like processes in classical conditioning. In Jones, M. R. (Ed.) Miami symposium on the prediction of behavior: aversive stimulation (pp. 9–31). Miami: University of Miami Press.
[39]
Rescorla, R. A. (1968). Probability of shock in the presence and absence of CS in fear conditioning. Journal of comparative physiological psychology, 66, 1–5.
[40]
Chris T. Allen, Thomas J. Madden, “A Closer Look at Classical Conditioning”, The Journal of Consumer Research, Vol. 12, No: 3, December 1985, pp. 301-315.
[41]
Elnora W. Stuart, Terence A. Shimp, Randall W. Enge, “Classical Conditioning of Consumer Attitudes: Four Experiments in an Advertising Context”, Journal of Consumer Research, Vol. 14, December 1987, pp. 334-349.
[42]
Chanthika Pornpitakpan, “A critical review of classical conditioning effects on consumer behavior”, Australasian Marketing Journal 20 (2012) 282–296.
[43]
Yi Fen Chen ve Jian Han Jiang, “Effects Of Classical and Operant Conditioning On Online Consumer Purchase and Repurchase Intention”, Effects Of Classical and Operant Conditioning On Online Consumer Purchase and Repurchase Intention.
[44]
Suping Liu, Yongsheng Ding, “A Classical Conditioning Model for Policy-based Management”, 2009 International Conference on Networks Security, Wireless Communications and Trusted Computing, DOI 10.1109/NSWCTC.2009.129, pp. 249-252.
[45]
Suping Liu, Yongsheng Ding, “An Adaptive Network Policy Management Framework Based on Classical Conditioning”, Proceedings of the 7th World Congress on Intelligent Control and Automation, June 25 - 27, 2008, Chongqing, China, pg. 3336-3340.
[46]
R. Prueckl, A.H. Taub, I. Herreros, R. Hogri, A. Magal, S.A. Bamford, A. Giovannucci, R. Ofek, Y. Shacham-Diamand, P.F.M.J. Verschure, M. Mintz, J. Scharinger, A. Silmon, C. Guger, “Behavioral Rehabilitation of the Eye Closure Reflex in Senescent Rats using a Real-Time Biosignal Acquisition System”, 33rd Annual International Conference of the IEEE EMBS, Boston, Massachusetts USA, August 30 - September 3, 2011, Pg. 4211-4214.
[47]
Guoshi Li, Gregory J. Quirk, and Satish S. Nair, “Modeling Acquisition and Extinction of Conditioned Fear in LA Neurons using Learning Algorithm”, Proceedings of the 2007 American Control Conference Marriott Marquis Hotel at Times Square New York City, USA, July 11-13, 2007, Pg. 552-557.
[48]
Susan Shi Yuan Li, Gavan P. McNally, “The conditions that promote fear learning: Prediction error and Pavlovian fear conditioning”, Neurobiology of Learning and Memory 108 (2014) 14–21.
[49]
Sven Benson, Joswin Kattoor, Jennifer S. Kullmann, Sarah Hofmann, Harald Engler, Michael Forsting, Elke R. Gizewski, Sigrid Elsenbruch, “Towards understanding sex differences in visceral pain: Enhanced reactivation of classically-conditioned fear in healthy women”, Neurobiology of Learning and Memory 109 (2014) 113–121.
[50]
Furze T. A., “The Application of Classical Conditioning to the Machine Learning of a Commonsense Knowledge of Visual Events”, PhD Thesis, The University of Leeds, School of Computing.
[51]
Elliot A. Ludvig, Marc G. Bellemare and Keir G. Pearson, “A Primer on Reinforcement Learning in the Brain: Psychological, Computational, and Neural Perspectives” in Eduardo Alonso, Esther Mondragón, “Computational Neuroscience for Advancing Artificial Intelligence: Models, Methods and Applications”, Medical Information Science Reference, Hershey • New York, 2011.
[52]
Dawson M. R. W., “Connectionism and Classical Conditioning”, A peer-reviewed monograph published by Comparative Cognition and Behavior Reviews on behalf of the Comparative Cognition Society. 2008 doi: 10.3819/ccbr.2008.30008.
[53]
Schmajuk N., “Mechanisms in Classical Conditioning: A Computational Approach”, Cambridge University Press, The Edinburgh Building, Cambridge CB2 8RU, UK, 2010.
[54]
Schmajuk N., “Computational Models of Conditioning”, Cambridge University Press, The Edinburgh Building, Cambridge CB2 8RU, UK, 2010.
[55]
Balkenius, Christian, and Jan Morén. "Computational models of classical conditioning: a comparative study." Proceedings of the fifth international conference on simulation of adaptive behavior on From animals to animats. Vol. 5. 1998.
[56]
Daniel L. Chester, “A Comparison of some Neural Network Models of Classical Conditioning”, 1990 IEEE, pp.1163-1168.
[57]
A. C. Courville, N. D. Daw, G. J. Gordon ve D. S. Touretzky, “Model Uncertainty in Classical Conditioning”, Advances in neural information processing systems. 2003.
[58]
Samuel J. Gershman & Yael Niv, “Exploring a latent cause theory of classical conditioning”, Learn Behav (2012) 40:255–268 DOI 10.3758/s13420-012-0080-8.
[59]
Anja Austermann, Seiji Yamada, “Learning to Understand Multimodal Rewards for Human-Robot-Interaction using Hidden Markov Models and Classical Conditioning”, 2008 IEEE Congress on Evolutionary Computation (CEC 2008), Pg. 4096-4103.
[60]
Christopher Johansson ve Anders Lansner, “An Associative Neural Network Model of Classical Conditioning”, Report from Studies of Artificial Neural Systems, (SANS), Numerical Analysis and Computer Science (Nada), Royal Institute of Technology (KTH) , S-100 44 STOCKHOLM, Sweden.
[61]
H. Hassan Ve M. Watan, “On Mathematical Analysis of Pavlovian Condltioning Learning Process Using Artificial Neural Network Model”, 10th Mediterranean Electrotechnical Conference, MEleCon 2000, Vol. II, Pg. 578-581.
[62]
Hiroyuki Ishii, Masaki Nakasuji, Motonori Ogura, Hiroyasu Miwa, Atsuo Takanishi, “Accelerating Rat’s Learning Speed Using a Robot - The robot autonomously shows rats its functions –”, Proceedings of the 2004 IEEE International Workshop on Robot and Human Interactive Communication, Kurashiki, Okayama Japan September 20-22,2004.
[63]
Sutton, Richard S., and Andrew G. Barto. "Toward a modern theory of adaptive networks: expectation and prediction." Psychological review 88.2 (1981): 135.
[64]
Wagner, A.R. (1981) SOP: A model of automatic memory processing in animal behavior. In Spear, N.E. and R.R. Miller, editors, Informatzon Processing in Animals: Memory Mechaieisms, volume 85, chapter 1, pages 5-44. Erlbaum, New Jersey.
[65]
Sutton, R.S. and A .G. Barto (1990) Time-derivative models of Pavlovian reinforcement. In Gabriel, A. and J. Moore, editors, Learning and Computational Neuroscience: Foundations and Adaptive Networks, chapter 12, pages 497-537. MIT Press, Cambridge, Mass and London.
[66]
Klopf, A.H. (1989) Classical conditioning: Phenomena predicted by a drive-reinforcement model of neural function. In Byrne, J.H. and W.O. Berry, editors, Neural Models of Placticity: Experimental and Theoretical Approaches, chapter 6, pages 94-103. Academic Press.
[67]
Christian Balkenius, Jan Mor´en, “Dynamics of a Classical Conditioning Model”, Autonomous Robots 7, 41–56 (1999).
[68]
Malaka, R., R. Lange, and M. Hammer (1995) A constant prediction model for classical conditioning. In Elser, N. and R. Menzel, editors, Learning and Memory: Proceedings of the 23rd Gottingen Neurobiology Conference, volume 1, page 75. Thieme-Verlag, Stuttgart, New York.
[69]
Klopf, A. H. (1988). A neuronal model of classical conditioning. 16, 2, 85–125.
[70]
John Y. Cheung, David C. Chance, and Asa Lawton, “An Examination of Real-Time Neuronal Models in a Classical Conditioning Framework”, 1992 IEEE, Pg. III-504 – III-509.
[71]
Ertugrul, O. F., & Tagluk, M. E. (2014, April). Learning with classical conditioning. In Signal Processing and Communications Applications Conference (SIU), 2014 22nd (pp. 927-930). IEEE. doi: 10.1109/SIU.2014.6830382.
[72]
Nestor A. Schmajuk and James J. DiCarlo, “Stimulus Configuration, Classical Conditioning, and Hippocampal Function”, Psychological Review, 1992, Vol. 99, No. 2,268-305.
[73]
Edgar H. Vogel, Maria E. Castro, Maria A. Saavedra, “Quantitative models of Pavlovian conditioning”, Brain Research Bulletin 63 (2004) 173–202.
[74]
Rainer Malaka, “Models of Classical Conditioning Models of Classical Conditioning”, Bulletin of Mathematical Biology (1999) 61, 33–83, Article No. bulm.1998.0074.
