A Spatial Cognitive Model that Integrates the Effects of Endogenous and Exogenous Information on the Hippocampus and Striatum

Citation: J. Huang, H. Y. Yang, X. G. Ruan, N. G. Yu, G. Y. Zuo, H. M. Liu. A spatial cognitive model that integrates the effects of endogenous and exogenous information on the hippocampus and striatum. International Journal of Automation and Computing. http://doi.org/10.1007/s11633-021-1286-z doi:  10.1007/s11633-021-1286-z
 Citation: Citation: J. Huang, H. Y. Yang, X. G. Ruan, N. G. Yu, G. Y. Zuo, H. M. Liu. A spatial cognitive model that integrates the effects of endogenous and exogenous information on the hippocampus and striatum. International Journal of Automation and Computing . http://doi.org/10.1007/s11633-021-1286-z

A Spatial Cognitive Model that Integrates the Effects of Endogenous and Exogenous Information on the Hippocampus and Striatum

Author Bio: Jing Huang received the Ph. D. degree in pattern recognition and intelligent system from Beijing University of Technology, China in 2016. Now she is an associate professor in Faculty of Information Technology, Beijing University of Technology, China. Her research interests include cognitive robotics, machine learning, and artificial Intelligence. E-mail: huangjing@bjut.edu.cn (Corresponding author) ORCID iD: 0000-0001-8804-7150 He-Yuan Yang received the B. Sc. degree in automation from North China University of Water Resources and Electric Power (NCWU), China in 2019. He is currently a master student in control science and engineering at Faculty of Information Technology of Beijing University of Technology, China. His research interest is cognitive robotics. E-mail: yangheyuan@emails.bjut.edu.cn Xiao-Gang Ruan received the Ph. D. degree in control science and engineering from Zhejiang University, China in 1992. Now he is a professor of Beijing University of Technology, and he is also as a director of Institute of Artificial Intelligent and Robots (IAIR). His research interests include automatic control, artificial intelligence, and intelligent robot. E-mail: adrxg@bjut.edu.cn Nai-Gong Yu received the B. Eng. degree in information processing display and recognition from Harbin Institute of Technology, China in 1989, the M. Eng. degree in control science and engineering from Shanghai Jiao Tong University, China in 1996, and the Ph. D. degree in pattern recognition and intelligent systems from Beijing University of Technology, China in 2005. He worked as a visiting scholar in University of Alberta, Canada in 2011. He is currently a professor with Faculty of Information Technology, Beijing University of Technology, China. His research interests include computational intelligence, intelligent systems and robotics. E-mail: yunaigong@bjut.edu.cn Guo-Yu Zuo received the Ph. D. degree in cybernetics from Beijing University of Technology, China in 2005. He is currently an associate professor and head of Intelligent Robot Laboratory of Beijing University of Technology, China. He has published over 50 journal and conference articles and achieved over 20 Chinese patents in artificial intelligence and robotics. His research interests include computational intelligence, robot learning, robot control, and human-robot interaction. E-mail: zuoguoyu@bjut.edu.cn Hao-Meng Liu received the B. Sc. degree in computer science and technology from Beijing University of Technology (BJUT), China in 2019. He is currently a master student in control engineering at Faculty of Information Technology of Beijing University of Technology, China. His research interest is industrial big data. E-mail: 15570122@emails.bjut.edu.cn
• Figure  1.  Architecture of the model

Figure  2.  Representation of visual information

Figure  3.  Neural network architecture of HPC

Figure  4.  Neural network connecting the hippocampus and striatum

Figure  5.  Flow chart of the working algorithm

Figure  6.  Experiment environment

Figure  7.  Paths in the experiments with different start points

Figure  8.  Paths in the experiments with different survival platforms

Figure  9.  Number of steps in the 25 experiments

Figure  10.  Paths of the agent in the advanced experiments

Figure  11.  Results of SARSA($\lambda$) algorithm

Figure  12.  Results of our model

Figure  13.  Results of different models after 50 trials

Figure  14.  Number of steps in the experiments with or without endogenous information

Figure  15.  Number of active place cells with different thresholds

Figure  16.  Number of updated Q values in 200 trials

•  [1] M. Contreras, T. Pelc, M. Llofriu, A. Weitzenfeld, J. M. Fellous. The ventral hippocampus is involved in multi-goal obstacle-rich spatial navigation. Hippocampus, vol. 28, no. 12, pp. 853–866, 2018. DOI:  10.1002/hipo.22993. [2] I. Ardiyanto, J. Miura. Time-space viewpoint planning for guard robot with chance constraint. International Journal of Automation and Computing, vol. 16, no. 4, pp. 475–490, 2019. DOI:  10.1007/s11633-018-1146-7. [3] A. D. Rast, S. V. Adams, S. Davidson, S. Davies, M. Hopkins, A. Rowley, A. B. Stokes, T. Wennekers, S. Furber, A. Cangelosi. Behavioral learning in a cognitive neuromorphic robot: An integrative approach. IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 12, pp. 6132–6144, 2018. DOI:  10.1109/TNNLS.2018.2816518. [4] N. Cazin, P. Scleidorovich, A. Weitzenfeld, P. F. Dominey. Real-time sensory–motor integration of hippocampal place cell replay and prefrontal sequence learning in simulated and physical rat robots for novel path optimization. Biological Cybernetics, vol. 114, no. 2, pp. 249–268, 2020. DOI:  10.1007/s00422-020-00820-2. [5] J. Nassour, T. D. Hoa, P. Atoofi, F. Hamker. Concrete action representation model: From neuroscience to robotics. IEEE Transactions on Cognitive and Developmental Systems, vol. 12, no. 2, pp. 272–284, 2020. DOI:  10.1109/TCDS.2019.2896300. [6] J. O′keefe, L. Nadel. The Hippocampus as a Cognitive Map, Oxford, UK: Oxford University Press, 1978. [7] T. Hafting, M. Fyhn, S. Molden, M. B. Moser, E. I. Moser. Microstructure of a spatial map in the entorhinal cortex. Nature, vol. 436, no. 7052, pp. 801–806, 2005. DOI:  10.1038/nature03721. [8] G. Tejera , M. Llofriu, A. Barrera, T. Alfredo. Bio-inspired robotics: a spatial cognition model integrating place cells, grid cells and head direction cells. Journal of Intelligent &Robotic Systems, vol. 91, no. 1, pp. 85–99, 2018. DOI:  10.1007/s10846-018-0852-2. [9] J. O′Keefe, N. Burgess. Geometric determinants of the place fields of hippocampal neurons. Nature, vol. 381, no. 6581, pp. 425–428, 1996. DOI:  10.1038/381425a0. [10] H. Wagatsuma, Y. Yamaguchi. Neural dynamics of the cognitive map in the hippocampus. Cognitive Neurodynamics, vol. 1, no. 2, pp. 119–141, 2007. DOI:  10.1007/s11571-006-9013-6. [11] Y. F. Zuo, F. Luo, C. L. Cui. Processing of spatial information by hippocampal place cells. Progress in Physiological Sciences, vol. 37, no. 1, pp. 6–10, 2006. (in Chinese) [12] J. W. Liu. Study on Navigation Methods of Intelligent Robots Based on Cognitive Map, Master dissertation, Northeastern University, China, 2015. (in Chinese) [13] E. I. Moser, M. B. Moser, P. Lipa, M. Newton, F. P. Houston, C. A. Barnes, B. L. McNaughton. A test of the reverberatory activity hypothesis for hippocampal “place” cells. Neuroscience, vol. 130, no. 2, pp. 519–526, 2005. DOI:  10.1016/j.neuroscience.2004.09.044. [14] C. Barry, C. Lever, R. Hayman, T. Hartley, S. Burton, J. O′Keefe, K. Jeffery, N. Burgess. The boundary vector cell model of place cell firing and spatial memory. Reviews in the Neurosciences, vol. 17, no. 1–2, pp. 71–97, 2006. DOI:  10.1515/revneuro.2006.17.1-2.71. [15] C. Lever, S. Burton, A. Jeewajee, J. O′Keefe, N. Burgess. Boundary vector cells in the subiculum of the hippocampal formation. Journal of Neuroscience, vol. 29, no. 31, pp. 9771–9777, 2009. DOI:  10.1523/JNEUROSCI.1319-09.2009. [16] R. M. Grieves, É. Duvelle, P. A. Dudchenko. A boundary vector cell model of place field repetition. Spatial Cognition &Computation, vol. 18, no. 3, pp. 217–256, 2018. DOI:  10.1080/13875868.2018.1437621. [17] E. T. Rolls, S. M. Stringer, T. Elliot. Entorhinal cortex grid cells can map to hippocampal place cells by competitive learning. Network:Computation in Neural Systems, vol. 17, no. 4, pp. 447–465, 2006. DOI:  10.1080/09548980601064846. [18] N. G. Yu, Y. S. Liao, X. G. Zheng. A spatial cognition model based on the selection mechanism of hippocampus place cells. Journal of Biomedical Engineering, vol. 37, no. 1, pp. 27–37, 2020. DOI:  10.7507/1001-5515.201901044. (in Chinese) [19] Y. Zhou, D. W. Wu. Location estimation model based on the transformation from grid cells to place cells. Journal of Electronics &Information Technology, vol. 39, no. 9, pp. 2272–2276, 2017. DOI:  10.11999/JEIT161284. (in Chinese) [20] R. S. Sutton, A. G. Barto. Reinforcement Learning: An Introduction, 2nd ed., Cambridge, UK: MIT Press, 2018. [21] F. Chersi, N. Burgess. The cognitive architecture of spatial navigation: Hippocampal and striatal contributions. Neuron, vol. 88, no. 1, pp. 64–77, 2015. DOI:  10.1016/j.neuron.2015.09.021. [22] C. M. A. Pennartz, R. Ito, P. F. M. J. Verschure, F. P. Battaglia, T. W. Robbins. The hippocampal–striatal axis in learning, prediction and goal-directed behavior. Trends in Neurosciences, vol. 34, no. 10, pp. 548–559, 2011. DOI:  10.1016/j.tins.2011.08.001. [23] T. V. Maia. Reinforcement learning, conditioning, and the brain: Successes and challenges. Cognitive,Affective,&Behavioral Neuroscience, vol. 9, no. 4, pp. 343–364, 2009. DOI:  10.3758/CABN.9.4.343. [24] T. Tanaka, K. Nishida, T. Kurita. Navigation of mobile robot using location map of place cells and reinforcement learning. Systems and Computers in Japan, vol. 38, no. 7, pp. 65–75, 2007. DOI:  10.1002/scj.20632. [25] F. Chersi, G. Pezzulo. Using hippocampal-striatal loops for spatial navigation and goal-directed decision-making. Cognitive processing, vol. 13, no. 1, pp. 125–129, 2012. [26] T. Madl, K. Chen, D. Montaldi, R. Trappl. Computational cognitive models of spatial memory in navigation space: A review. Neural Networks, vol. 65, pp. 18–43, 2015. DOI:  10.1016/j.neunet.2015.01.002. [27] S. Doboli, A. A. Minai, P. J. Best. A computational model of the interaction between external and internal cues for the control of hippocampal place cells. Neurocomputing, vol. 52–54, pp. 371–379, 2003. DOI:  10.1016/S0925-2312(02)00843-3. [28] A. Aggarwal. The sensori-motor model of the hippocampal place cells. Neurocomputing, vol. 185, pp. 142–152, 2016. DOI:  10.1016/j.neucom.2015.12.044. [29] T. Madl, S. Franklin, K. Chen, D. Montaldi, R. Trappl. Bayesian integration of information in hippocampal place cells. PLoS One, vol. 9, no. 3, Article number e89762, 2014. DOI:  10.1371/journal.pone.0089762. [30] A. Arleo, W. Gerstner. Spatial cognition and neuro-mimetic navigation: A model of hippocampal place cell activity. Biological Cybernetics, vol. 83, no. 3, pp. 287–299, 2000. DOI:  10.1007/s004220000171. [31] M. G. Packard. Anxiety, cognition, and habit: A multiple memory systems perspective. Brain Research, vol. 1293, pp. 121–128, 2009. DOI:  10.1016/j.brainres.2009.03.029. [32] N. Arias, M. Méndez, J. L. Arias. Brain networks underlying navigation in the Cincinnati water maze with external and internal cues. Neuroscience Letters, vol. 576, pp. 68–72, 2014. DOI:  10.1016/j.neulet.2014.05.064. [33] D. Sukumar, S. Chakravarthy. A computational neuromotor model of the role of basal ganglia and hippocampus in spatial navigation. In Proceedings of the 22th International Conference on Artificial Neural Networks, Springer, Berlin, Heidelberg, Germany, pp. 216-221, 2010. [34] L. F. Hu, K. R. Hao, X. Cai, L. Chen. A spatial cognitive cells inspired goal-directed navigation model. In Proceedings of IEEE International Conference on Artificial Intelligence and Computer Applications, Dalian, China, pp. 211–215, 2019. [35] J. O′Keefe, J. Dostrovsky. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Research, vol. 34, no. 1, pp. 171–175, 1971. DOI:  10.1016/0006-8993(71)90358-1. [36] R. G. M. Morris, P. Garrud, J. N. P. Rawlins, J. O′Keefe. Place navigation impaired in rats with hippocampal lesions. Nature, vol. 297, no. 5868, pp. 681–683, 1982. DOI:  10.1038/297681a0. [37] T. Kulvicius, M. Tamosiunaite, J. Ainge, P. Dudchenko, F. Wörgötter. Odor supported place cell model and goal navigation in rodents. Journal of Computational Neuroscience, vol. 25, no. 3, pp. 481–500, 2008. DOI:  10.1007/s10827-008-0090-x. [38] J. J. Knierim, H. S. Kudrimoti, B. L. McNaughton. Place cells, head direction cells, and the learning of landmark stability. Journal of Neuroscience, vol. 15, no. 3, pp. 1648–1659, 1995. DOI:  10.1523/JNEUROSCI.15-03-01648.1995. [39] A. J. Hill, P. J. Best. Effects of deafness and blindness on the spatial correlates of hippocampal unit activity in the rat. Experimental Neurology, vol. 74, no. 1, pp. 204–217, 1981. DOI:  10.1016/0014-4886(81)90159-X. [40] G. J. Quirk, R. U. Muller, J. L. Kubie. The firing of hippocampal place cells in the dark depends on the rat′s recent experience. Journal of Neuroscience, vol. 10, no. 6, pp. 2008–2017, 1990. DOI:  10.1523/JNEUROSCI.10-06-02008.1990. [41] R. Moussa, B. Poucet, M. Amalric, F. Sargolini. Contributions of dorsal striatal subregions to spatial alternation behavior. Learning &Memory, vol. 18, no. 7, pp. 444–451, 2011. DOI:  10.1101/lm.2123811. [42] C. H. Donahue, M. Liu, A. C. Kreitzer. Distinct value encoding in striatal direct and indirect pathways during adaptive learning. [Online], Available: https://www.biorxiv.org/content/biorxiv/early/2018/03/07/277855.full.pdf, May, 2020. [43] E. L. Thorndike, Jelliffe. Animal intelligence: Experimental studies. Journal of Nervous &Mental Disease, vol. 39, no. 5, Article number 357, 1912. [44] J. Huang, X. G. Ruan, N. G. Yu, Q. W. Fan, J. M. Li, J. X. Cai. A cognitive model based on neuromodulated plasticity. Computational Intelligence and Neuroscience, vol. 2016, Article number 4296356, 2016. DOI:  10.1155/2016/4296356. [45] D. Sheynikhovich, R. Chavarriaga, T. Strösslin, W. Gerstner. Spatial representation and navigation in a bio-inspired robot. In Proceedings of Biomimetic Neural Learning for Intelligent Robots: Intelligent Systems, Cognitive Robotics, and Neuroscience, Springer, Berlin, Heidelberg, Germany, pp. 245−264, 2005. [46] M. Y. Zhang, G. H. Tian, C. C. Li, J. Gong. Learning to transform service instructions into actions with reinforcement learning and knowledge base. International Journal of Automation and Computing, vol. 15, no. 5, pp. 582–592, 2018. DOI:  10.1007/s11633-018-1128-9.
•  [1] Shuo Huang, Wei Shao, Mei-Ling Wang, Dao-Qiang Zhang.  fMRI-based Decoding of Visual Information from Human Brain Activity: A Brief Review . International Journal of Automation and Computing, doi: 10.1007/s11633-020-1263-y [2] Wei-Ning Wang, Qi Li, Liang Wang.  Robust Object Tracking via Information Theoretic Measures . International Journal of Automation and Computing, doi: 10.1007/s11633-020-1235-2 [3] Negin Alborzi, Fereshteh Poorahangaryan, Homayoun Beheshti.  Spectral-spatial Classification of Hyperspectral Images Using Signal Subspace Identification and Edge-preserving Filter . International Journal of Automation and Computing, doi: 10.1007/s11633-019-1188-5 [4] Siuly Siuly, Varun Bajaj, Abdulkadir Sengur, Yanchun Zhang.  An Advanced Analysis System for Identifying Alcoholic Brain State Through EEG Signals . International Journal of Automation and Computing, doi: 10.1007/s11633-019-1178-7 [5] Tian-Jie Zhang.  Unmanned Aerial Vehicle Formation Inspired by Bird Flocking and Foraging Behavior . International Journal of Automation and Computing, doi: 10.1007/s11633-017-1111-x [6] Jhon A. Isaza, Hector A. Botero, Hernan Alvarez.  State Estimation Using Non-uniform and Delayed Information: A Review . International Journal of Automation and Computing, doi: 10.1007/s11633-017-1106-7 [7] Biao Chang, Tong Xu, Qi Liu, En-Hong Chen.  Study on Information Diffusion Analysis in Social Networks and Its Applications . International Journal of Automation and Computing, doi: 10.1007/s11633-018-1124-0 [8] Huan-Zhao Chen, Guo-Hui Tian, Guo-Liang Liu.  A Selective Attention Guided Initiative Semantic Cognition Algorithm for Service Robot . International Journal of Automation and Computing, doi: 10.1007/s11633-018-1139-6 [9] Hong Qiao, Hong Zhang, Florian Röhrbein.  Editorial for Special Issue on Human-inspired Computing . International Journal of Automation and Computing, doi: 10.1007/s11633-017-1097-4 [10] Sun-Chun Zhou, Rui Yan, Jia-Xin Li, Ying-Ke Chen, Huajin Tang.  A Brain-inspired SLAM System Based on ORB Features . International Journal of Automation and Computing, doi: 10.1007/s11633-017-1090-y [11] Jinhua She, Hiroshi Hashimoto, Min Wu.  Reduced-order Modeling of Human Body for Brain Hypothermia Treatment . International Journal of Automation and Computing, doi: 10.1007/s11633-016-0961-y [12] Yi-Quan Song, Lei Niu, Long He, Rui Wang.  A Grid-based Graph Data Model for Pedestrian Route Analysis in a Micro-spatial Environment . International Journal of Automation and Computing, doi: 10.1007/s11633-016-0979-1 [13] Sepehr Keykhaie, Saleh Yousefi, Mehdi Dehghan.  Modeling of Propagation of Road Hazard Information in Sparse Vehicular ad hoc Networks . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0860-z [14] Pradeep Kumar.  Computation of Resonant Frequency of Gap-coupled Ring Microstrip Antennas . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0814-5 [15] .  Methods for Controlling the Authenticity of Textual Information Transfer on the Basis of Statistical and Structural Redundancy . International Journal of Automation and Computing, doi: 10.1007/s11633-012-0675-8 [16] Biliana Alexandrova-Kabadjova, Edward Tsang, Andreas Krause.  Market Structure and Information in Payment Card Markets . International Journal of Automation and Computing, doi: 10.1007/s11633-011-0593-1 [17] Jing Sun,  Ying-Jie Xing.  An Effective Image Retrieval Mechanism Using Family-based Spatial Consistency Filtration with Object Region . International Journal of Automation and Computing, doi: 10.1007/s11633-010-0023-9 [18] Tetsuro Morimoto, Tohru Mihashi, Katsushi Ikeuchi.  Color Restoration Method Based on Spectral Information Using Normalized Cut . International Journal of Automation and Computing, doi: 10.1007/s11633-008-0226-5 [19] Woong Choi, Tadao Isaka, Mamiko Sakata, Seiya Tsuruta, Kozaburo Hachimura.  Quantification of Dance Movement by Simultaneous Measurement of Body Motion and Biophysical Information . International Journal of Automation and Computing, doi: 10.1007/s11633-007-0001-z [20] Jin-Dong Liu,  Huosheng Hu.  Biologically Inspired Behaviour Design for Autonomous Robotic Fish . International Journal of Automation and Computing, doi: 10.1007/s11633-006-0336-x

计量
• 文章访问数:  5
• HTML全文浏览量:  15
• PDF下载量:  3
• 被引次数: 0
出版历程
• 收稿日期:  2020-10-08
• 录用日期:  2021-01-28
• 网络出版日期:  2021-03-20

A Spatial Cognitive Model that Integrates the Effects of Endogenous and Exogenous Information on the Hippocampus and Striatum

English Abstract

Citation: J. Huang, H. Y. Yang, X. G. Ruan, N. G. Yu, G. Y. Zuo, H. M. Liu. A spatial cognitive model that integrates the effects of endogenous and exogenous information on the hippocampus and striatum. International Journal of Automation and Computing. http://doi.org/10.1007/s11633-021-1286-z doi:  10.1007/s11633-021-1286-z
 Citation: Citation: J. Huang, H. Y. Yang, X. G. Ruan, N. G. Yu, G. Y. Zuo, H. M. Liu. A spatial cognitive model that integrates the effects of endogenous and exogenous information on the hippocampus and striatum. International Journal of Automation and Computing . http://doi.org/10.1007/s11633-021-1286-z

/

• 分享
• 用微信扫码二维码

分享至好友和朋友圈