SWAM: Workload Automatic Mapper for SNN

doi:10.3778/j.issn.1673-9418.2010056

Abstract

Abstract:

In order to meet the computing requirements of large-scale spiking neural network (SNN), neuromorphic computing systems usually need to adopt large-scale parallel computing platforms. Therefore, how to quickly deter-mine a reasonable number of computing nodes for the SNN workload (that is, how to properly map the workload to the computing platform) to obtain the best performance, better power consumption and other indicators has become one of the key issues that a neuromorphic computing system needs to solve. Firstly, this paper analyzes the SNN workload characteristics and establishes a calculation model for it. Then for the NEST simulator, this paper further instantiates SNN load model of storage, calculation and communication. Finally, this paper designs and implements a NEST-based workload automatic mapper for SNN (SWAM). SWAM can automatically calculate the mapping result and complete the mapping, avoiding the extremely time-consuming manual trial process of workload mapping. SNN typical applications are run on three different computing platforms, ARM+FPGA, ARM, and PC clusters, and the mapping results of SWAM and LM (Levenberg-Marquardt) algorithm fitting and measured are compared. Experi-mental results show that the average mapping accuracy of SWAM reaches 98.833%. Compared with the LM method and the measured mapping, SWAM has absolute advantage in time cost.

Key words: spiking neural network (SNN), workload mapping, PYNQ clusters, field programmable gate array (FPGA) acceleration, NEST simulator

摘要：

为了满足大规模脉冲神经网络（SNN）的计算需求，类脑计算系统通常需要采用大规模并行计算平台。因此，如何快速为SNN工作负载确定合理的计算节点数（即如何把工作负载合理映射到计算平台上）以获得最佳的性能、功耗等指标就成为类脑计算系统需解决的关键问题之一。首先分析了SNN工作负载特性并为其建立起计算模型；然后针对NEST类脑仿真器，进一步实例化了SNN的内存、计算和通信负载模型；最终设计并实现了一种基于NEST的SNN工作负载自动映射器（SWAM）。SWAM可以自动计算出映射结果并完成映射，避免了极其耗时的工作负载映射手动试探过程。在ARM+FPGA、纯ARM、PC集群三种不同的计算平台上运行SNN典型应用，并比较SWAM、LM算法拟合和实测的映射结果。实验结果表明：SWAM的平均映射准确率达到98.833%，与LM方法与实测映射相比，SWAM具有绝对的时间代价优势。

关键词: 脉冲神经网络（SNN）, 工作负载映射, PYNQ集群, 现场可编程逻辑门阵列（FPGA）加速, NEST仿真器

YU Gongjian, ZHANG Lufei, LI Peiqi, HUA Xia, LIU Jiahang, CHAI Zhilei, CHEN Wenjie. SWAM: Workload Automatic Mapper for SNN[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(9): 1641-1657.

郁龚健, 张鲁飞, 李佩琦, 华夏, 刘家航, 柴志雷, 陈闻杰. SWAM：SNN工作负载自动映射器[J]. 计算机科学与探索, 2021, 15(9): 1641-1657.

References

[1] MAASS W. Networks of spiking neurons: the third genera-tion of neural network models[J]. Neural Networks, 1997, 10(9): 1659-1671.
[2] MA D, SHEN J, GU Z, et al. Darwin: a neuromorphic hardware co-processor based on spiking neural networks[J]. Journal of Systems Architecture, 2017, 77: 43-51.
[3] FURBER S B, GALLUPPI F, TEMPLE S, et al. The spin-naker project[J]. Proceedings of the IEEE, 2014, 102(5): 652-665.
[4] DAVIES M, SRINIVASA N, LIN T H, et al. Loihi: a neuro-morphic manycore processor with on-chip learning[J]. IEEE Micro, 2018, 38(1): 82-99.
[5] MEROLLA P A, ARTHUR J V, ALVAREZ-ICAZA R, et al.A million spiking-neuron integrated circuit with a scalable communication network and interface[J]. Science, 2014, 345(6197): 668-673.
[6] SCHEMMEL J, KRIENER L, MüLLER P, et al. An acce-lerated analog neuromorphic hardware system emulating NMDA-and calcium-based non-linear dendrites[C]//Proceed-ings of the 2017 International Joint Conference on Neural Networks, Anchorage, May 14-19, 2017. Piscataway: IEEE,2017: 2217-2226.
[7] BENJAMIN B V, GAO P, MCQUINN E, et al. Neurogrid: a mixed-analog-digital multichip system for large-scale ne-ural simulations[J]. Proceedings of the IEEE, 2014, 102(5): 699-716.
[8] POTJANS T C, DIESMANN M. The cell-type specific cor-tical microcircuit: relating structure and activity in a full-scale spiking network model[J]. Cerebral Cortex, 2014, 24(3): 785-806.
[9] VAN ALBADA S J, ROWLEY A G, SENK J, et al. Perfor-mance comparison of the digital neuromorphic hardware SpiNNaker and the neural network simulation software NEST for a full-scale cortical microcircuit model[J]. Frontiers in Neuroscience, 2018, 12: 291.
[10] KNIGHT J C, NOWOTNY T. GPUs outperform current HPC and neuromorphic solutions in terms of speed and energy when simulating a highly-connected cortical model[J]. Fron-tiers in Neuroscience, 2018, 12: 941.
[11] SUSANNE K, POTJANS T C, EPPLER J M, et al. Meet-ing the memory challenges of brain-scale network simula-tion[J]. Frontiers in Neuroinformatics, 2012, 5: 35.
[12] SCHENCK W, ADINETZ A V, ZAYTSEV Y V, et al. Performance model for large-scale neural simulations with NEST[C]//Proceedings of the Supercomputing 2014, New Orleans, Nov 16-21, 2014: 1-2.
[13] BRETTE R, RUDOLPH M, CARNEVALE T, et al. Simul-ation of networks of spiking neurons: a review of tools and strategies[J]. Journal of Computational Neuroscience, 2007, 23(3): 349-398.
[14] KASS R E, AMARI S I, ARAI K, et al. Computational neuro-science: mathematical and statistical perspectives[J]. Annual Review of Statistics and Its Application, 2018, 3: 1-37.
[15] CAO Y Q, CHEN Y, KHOSLA D. Spiking deep convolu-tional neural networks for energy-efficient object recogni-tion[J]. International Journal of Computer Vision, 2015, 113(1): 54-66.
[16] DIEHL P U, COOK M. Unsupervised learning of digit recog-nition using spike-timing-dependent plasticity[J]. Frontiers in Computational Neuroscience, 2015, 9: 99.
[17] GERSTNER W, KISTLER W M, NAUD R, et al. Neuronal dynamics: from single neurons to networks and models of cognition[M]. New York: Cambridge University Press, 2014.
[18] BI G Q, POO M M. Synaptic modifications in cultured hi-ppocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type[J]. Journal of Neuro-science, 2012, 18(24): 10464-10472.
[19] FIDJELAND A K, SHANAHAN M P. Accelerated simu-lation of spiking neural networks using GPUs[C]//Proceed-ings of the 2010 International Joint Conference on Neural Networks, Barcelona, Jul 18-23, 2010. Piscataway: IEEE, 2010: 1-8.
[20] RUSSELL A, ORCHARD G, DONG Y, et al. Optimization methods for spiking neurons and networks[J]. IEEE Tran-sactions on Neural Networks, 2010, 21(12): 1950-1962.
[21] GEWALTIG M O, DIESMANN M. NEST (neural simula-tion tool)[J]. Scholarpedia, 2007, 2(4): 1430.
[22] STIMBERG M, GOODMAN D, BENICHOUX V, et al. Equation-oriented speci?cation of neural models for simula-tions[J]. Frontiers in Neuroinformatics, 2014, 8: 1-15.
[23] HINES M, CARNEVALE N. The NEURON simulation en-vironment[J]. Neural Computation, 2014, 9(6): 1179-1209.
[24] LI K, ZHANG L F, ZHANG X W, et al. Design of spiking neural network simulator based on FPGA cluster[J]. Com-puter Engineering, 2020, 46(10): 201-209.
李康, 张鲁飞, 张新伟, 等. 基于FPGA集群的脉冲神经网络仿真器[J]. 计算机工程, 2020, 46(10): 201-209.
[25] ZHANG X W, LI K, YU G J, et al. Research and imple-mentation of accelerating neuromorphic computing based on ZYNQ cluster[J]. Computer Engineering and Applica-tions, 2020, 56(21): 65-71.
张新伟, 李康, 郁龚健, 等. 基于ZYNQ集群的神经形态计算加速研究与实现[J]. 计算机工程与应用, 2020, 56(21): 65-71.
[26] MORRISON A, AERTSEN A, DIESMANN M. Spike-timing-dependent plasticity in balanced random networks[J]. Neural Computation, 2007, 19(6): 1437-1467.
[27] KHERADPISHEH S R, GANJTABESH M, THORPE S J, et al. STDP-based spiking deep convolutional neural net-works for object recognition[J]. Neural Networks, 2018, 99: 56-67.