Abstract: In recent years, deep belief network (DBN) has been widely used, but there are few detailed studies on how to dynamically determine its structure in the existing literature. In this paper, a dynamic DBN (DDBN) model using dynamic growing and pruning algorithm is proposed, which can effectively optimize the DBN network structure. The DDBN can automatically determine its structure using dynamic growing and pruning algorithm instead of artificial experience. Firstly, the structure of DDBN is constructed automatically by changing the number of both the hidden layers and hidden neurons during the training process. This is implemented by automatic growing and pruning algorithm, which depends on the weight distance (WD) and the standard deviation of activation probability of hidden neurons, and the energy function of the entire network. Secondly, DDBN is able to adjust the weights in the dynamic process and is helpful to improve the network performances. Finally, in order to verify the validity of the proposed DDBN, the proposed DDBN has been tested on three image benchmark data sets, including MNIST, USPS and CIFAR-10. Experimental results show that DDBN has better performances than some existing DBN structure adjustment methods.

Key words: deep learning, dynamic deep belief network (DDBN), dynamic growing and pruning algorithm, network structure optimization

摘要： 近年来深度信念网络（DBN）得到了广泛的应用，但在现有文献中很少有关于如何动态确定其结构的详细研究。提出了一种使用动态增减枝算法的DBN模型（DDBN），可以有效地优化DBN的网络结构。DDBN可以使用动态增减枝算法而不是人工实验来自动确定其结构。首先，在训练过程中通过改变隐藏层层数和隐藏层神经元的数量，自动构建DDBN的结构，这是通过动态增减枝算法实现的。该算法依赖于隐藏层神经元的权重距离（WD）和激活概率的标准差以及整个网络的能量函数。其次，DDBN能够在动态过程中调整权重，有助于提高网络性能。最后，为了验证DDBN的有效性，将DDBN在MNIST、USPS和CIFAR-10三个基准图像数据集上进行了测试。实验结果表明，DDBN比现有的一些DBN结构调整方法具有更好的性能。

关键词: 深度学习, 动态深度信念网络, 动态增减枝算法, 网络结构优化