关键词: 深度学习, 动态深度信念网络, 动态增减枝算法, 网络结构优化

Abstract: In recent years, deep belief network (DBN) has been widely used, but there are few detailed studies on how to dynamically determine its structure in the existing literature. In this paper, a dynamic DBN (DDBN) model using dynamic growing and pruning algorithm is proposed, which can effectively optimize the DBN network structure. The DDBN can automatically determine its structure using dynamic growing and pruning algorithm instead of artificial experience. Firstly, the structure of DDBN is constructed automatically by changing the number of both the hidden layers and hidden neurons during the training process. This is implemented by automatic growing and pruning algorithm, which depends on the weight distance (WD) and the standard deviation of activation probability of hidden neurons, and the energy function of the entire network. Secondly, DDBN is able to adjust the weights in the dynamic process and is helpful to improve the network performances. Finally, in order to verify the validity of the proposed DDBN, the proposed DDBN has been tested on three image benchmark data sets, including MNIST, USPS and CIFAR-10. Experimental results show that DDBN has better performances than some existing DBN structure adjustment methods.

Key words: deep learning, dynamic deep belief network (DDBN), dynamic growing and pruning algorithm, network structure optimization