关键词: 加速器, 卷积神经网络（CNN）, 轻量化神经元计算单元（NCU）, 空间并行加速阵列（MSNAP）, 分支卷积量化

Abstract: Recently, the Convolutional Neural Network (CNN) has been widely used in image classification, object detection and recognition, and natural language processing. Due to the increasing complexity and scale of CNN, hardware realization is facing greater challenges, particularly for embedded systems with low power and latency requirements. Among them, the high throughput requirement coming from processing hundreds of filters in high-dimensional convolutions is foremost. Although high-parallel compute arrays address the high throughput requirement, energy consumption from the huge amount of data movement and resource consumption from high-parallel arrays remain unacceptable for embedded application scenarios，Not to mention the increased complexity in control that comes with it. This paper aims to optimize the energy efficiency of accelerators by analyzing the key factors that determine system energy efficiency. Therefore, focusing on reducing energy consumption and resource consumption of the system, this paper proposes a high-performance CNN accelerator, called MSNAP. This is realized by 128×128 high-parallel compute arrays and bases on a unified quantization method for the entire network with extremely low bit-width. To ensure that the data of input images and last layer are the same bit-width as the middle layers, we adopt thermometer codes and Branch Convolution Quantization methods, that allows integrating all memory on-chip and make the implementation of a large-scale array easier. MSNAP features an efficiencies lightweight computation neuron which composes of 1152 multiplicative cells. In addition to compressed memory storage, the unified quantization method simplifies multiplications to Multiplexer, which drastically reduces resource consumption. A weight-stationary, data-parallel dataflow and the optimization of the pooling layer improve data utilizations, which minimize data movement energy consumption of MSNAP. To evaluate the efficiencies of this design, and demonstrate the high-parallel compute arrays, a CNN chip in 22-nm CMOS was realized. Experiments show that at the frequency of 20MHz, the chip offers a peak throughput of 10.54 Tera Operations Per second(TOPS) and the efficiencies up to 64.317TOPS/W, the corresponding efficiencies amounts to a 5x improvement over the previous benchmark on CIFAR-10. Meanwhile, the design operates at 60frames/s (FPS) on YOLO, fully meets the needs of embedded applications.

Key words: Accelerator, Convolutional Neural Networks (CNN), lightweight computation neuron (NCU), high-parallel array (MSNAP), Branch convolution quantization