PEST: Energy-Efficient NEST Brain-Like Simulator Implemented by PYNQ Cluster

doi:10.3778/j.issn.1673-9418.2011047

Abstract

Abstract:

Large-scale brain-like simulation with high performance and low power consumption is one of the most challenging problems in brain-like computing. At present, the implementation of brain-like computing is mainly divided into hardware implementation and software implementation. Dedicated brain-like computing chips and systems implemented by hardware can provide better energy efficiency indicators, but they are costly and poorly adaptable. Software-based simulation (such as NEST) has good availability but has the problem of slow computing speed. If the two implementation methods are combined, through the software and hardware co-design, to ensure a good application ecology while obtaining higher computing energy efficiency, this paper proposes a high-energy-efficiency implemen-tation (PEST) of the NEST brain-like simulator based on the FPGA heterogeneous platform PYNQ cluster. By building a large-scale PYNQ cluster, it designs software and hardware data interaction interfaces to realize a scalable brain-like computing system based on the NEST simulator, designs FPGA hardware circuits for IAF neurons, and uses MPI distributed computing to improve NEST computing efficiency. The experimental results show that, for different computing models, under the optimal adaptation of PYNQ cluster, the performance of the neuron update part on PEST is improved by more than 4.6 times compared with AMD 3600X, and by more than 7.5 times compared with Xeon 2620. PEST's updated energy efficiency is more than 5.3 times higher than that of 3600X and 7.9 times higher than that of Xeon 2620.

Key words: brain-like computing, spiking neural networks, NEST simulator, field programmable gate array (FPGA), PYNQ framework

摘要：

高性能且低功耗地进行大规模类脑仿真是类脑计算所需解决的最具挑战的问题之一。目前类脑计算的实现方式主要分为硬件实现和软件实现两种。通过硬件实现的专用类脑计算芯片与系统可以提供更佳的能效指标，但代价高、适应性差；基于软件方式的仿真（如NEST）拥有完整的应用生态，可用性好但存在计算速度慢的问题。如果将两种实现方式相结合，通过软硬件协同设计，可以在保证良好应用生态的同时获得更高的计算能效，提出了一种基于FPGA异构平台PYNQ集群的NEST类脑仿真器的高能效实现（PEST）。通过构建大规模PYNQ集群，设计软硬件数据交互接口实现基于NEST仿真器的规模可伸缩类脑计算系统，针对IAF神经元进行FPGA硬件电路设计，利用MPI分布式计算等方式提升了NEST计算效率。实验结果表明：针对不同的计算模型，在PYNQ集群最佳适配情况下，PEST上神经元更新部分的性能相比AMD 3600X提升超过4.6倍，相比Xeon 2620提升超过7.5倍；PEST的更新能效比相比3600X提升超过5.3倍，相比Xeon 2620提升超过7.9倍。

关键词: 类脑计算, 脉冲神经网络, NEST仿真器, 现场可编程门阵列（FPGA）, PYNQ框架

LI Peiqi, YU Gongjian, HUA Xia, LIU Jiahang, CHAI Zhilei. PEST: Energy-Efficient NEST Brain-Like Simulator Implemented by PYNQ Cluster[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(11): 2127-2141.

李佩琦，郁龚健，华夏，刘家航，柴志雷. PEST：由PYNQ集群实现的高能效NEST类脑仿真器[J]. 计算机科学与探索, 2021, 15(11): 2127-2141.

References

[1] MARCUS G. Deep learning: a critical appraisal[J]. arXiv:1801.00631, 2018.
[2] FURBER S B, LESTER D R, PLANA L A, et al. Overview of the SpiNNaker system architecture[J]. IEEE Transactions on Computers, 2013, 62(12): 2454-2467.
[3] MAAS W. Networks of spiking neurons: the third generation of neural network models[J]. Neural Networks, 1997, 10(9): 1659-1671.
[4] MIKAITIS M, LESTER D R, SHANG D L, et al. Appro-ximate fixed-point elementary function accelerator for the SpiNNaker-2 neuromorphic chip[C]//Proceedings of the 2018 IEEE Symposium on Computer Arithmetic, Jun 25-27, 2018. Piscataway: IEEE, 2018: 37-44.
[5] FURBER S B, GALLUPPI F, TEMPLE S, et al. The SpiN-Naker project[J]. Proceedings of the IEEE, 2014, 102(5): 652-665.
[6] YAKOPCIC C, RAHMAN N, ATAHARY T, et al. Solving constraint satisfaction problems using the Loihi spiking neuromorphic processor[C]//Proceedings of the 2020 Design, Automation & Test in Europe Conference & Exhibition, Grenoble, Mar 9-13, 2020. Piscataway: IEEE, 2020: 1079-1084.
[7] DENG L, WANG G R, LI G Q, et al. Tianjic: a unified and scalable chip bridging spike-based and continuous neural computation[J]. IEEE Journal of Solid-State Circuits, 2020, 55(8): 2228-2246.
[8] WU C H. Brain-like research: building a superbrain for human[J]. Journal of Zhejiang University (Engineering Science), 2020, 54(3): 425-426.
吴朝晖. 类脑研究：为人类构建超级大脑[J]. 浙江大学学报(工学版), 2020, 54(3): 425-426.
[9] POTJANS T C, DIESMANN M. The cell-type specific cortical microcircuit: relating structure and activity in a full-scale spiking network model[J]. Cerebral Cortex, 2014, 24(3): 785-806.
[10] VAN ALBADA S J, ROWLEY A G, JOHANNA S, et al. Performance comparison of the digital neuromorphic hardware SpiNNaker and the neural network simulation software NEST for a full-scale cortical microcircuit model[J]. Frontiers in Neuroscience, 2018, 12: 291.
[11] KNIGHT J C, NOWOTNY T. GPUs outperform current HPC and neuromorphic solutions in terms of speed and energy when simulating a highly-connected cortical model[J]. Frontiers in Neuroscience, 2018, 12: 941.
[12] GOODMAN D F, BRETTE R. The briansimulator[J]. Fron-tiers in Neuroscience, 2009, 3(2): 192.
[13] GEWALTIG M M O, DIESMANN M. NEST (neural simu-lation tool)[J]. Scholarpedia, 2007, 2(4): 1430.
[14] HAZAN H, SAUNDERS D J, KHAN H, et al. BindsNET: a machine learning-oriented spiking neural networks library in Python[J]. Frontiers in Neuroinformatics, 2018, 12: 89.
[15] KUNKEL S, SCHENCK W. The nest dry-run mode: efficient dynamic analysis of neuronal network simulation code[J]. Frontiers in Neuroinformatics, 2017, 11: 40.
[16] IZHIKEVICH E M. Which model to use for cortical spiking neurons?[J]. IEEE Transactions on Neural Networks, 2004, 15(5): 1063-1070.
[17] EPPLER J M, HELIAS M, MULLER E, et al. PyNEST: a convenient interface to the NEST simulator[J]. Frontiers in Neuroinformatics, 2009, 2: 12.
[18] YAMAURA H, IGARASHI J, YAMAZAKI T. Simulation of a human-scale cerebellar network model on the K com-puter[J]. Frontiers in Neuroinformatics, 2020, 14: 16.
[19] “pynq. io” [EB/OL]. [2020-08-25]. http://www.pynq.io.
[20] STORNAIUOLO L, SANTAMBROGIO M D, SCIUTO D. On how to efficiently implement deep learning algorithms on PYNQ platform[C]//Proceedings of the 2018 IEEE Computer Society Annual Symposium on VLSI, Hong Kong, China, Jul 8-11, 2018. Washington: IEEE Computer Society, 2018: 587-590.
[21] SCHELLE G. Keynote 1-growing the ReConFig commu-nity through python, zynq and hardware overlays[C]//Procee-dings of the 2016 International Conference on ReConFigur-able Computing and FPGAs, Cancun, Nov 30-Dec 2, 2016. Piscataway: IEEE, 2016: 1.
[22] KUNKEL S, SCHMIDT M, EPPLER J M, et al. Spiking network simulation code for petascale computers[J]. Frontiers in Neuroinformatics, 2014, 8: 78.
[23] ROTTER S, DIESMANN M. Exact digital simulation of time-invariant linear systems with applications to neuronal modeling[J]. Biological Cybernetics, 1999, 81(5/6): 381-402.
[24] TSODYKS M, UZIEL A, MARKRAM H. Synchrony genera-tion in recurrent networks with frequency-dependent synapses[J]. The Journal of Neuroscience, 2000, 20(1): RC50.
[25] MASQUELIER T, THORPE S J. Unsupervised learning of visual features through spike timing dependent plasticity[J]. PLoS Computational Biology, 2007, 3(2): e31.