关键词: 深度学习, 神经网络, 网络压缩, 模型加速, 剪枝

Abstract: With the development of deep learning, the structure of deep neural network is becoming more and more complicated, which makes the depth, size and computation increase, as well as memory cost and GPU cost, so it is difficult to apply the model to mobile terminal and embedded devices like auto-driving which have limited resources and high real-time request. Lightening network is the future, building small and efficient neural network has been a hot spot. By studying the propagation mode of weight update of neural network, this paper proposes a simple but useful way to compress the net. For the convolution net, this paper applies the separable convolution to speed the process and evaluate it at a standard 5 layer network. As for full-connect layer, this paper uses a pruning method based on weight similarity, which can prune more than 90% full connect layer units without obviously accuracy decline on mnist dataset and cifar10 dataset.

Key words: deep learning, neural network, net compression, model accelerate, units pruning