Abstract: Deep belief network (DBN) is trained through layer-by-layer unsupervised learning，but it is easy to generate a large amount of redundant information in the training process and affecting the feature extraction ability. In order to make the model more explanatory and discriminative, a penalty regularization term is introduced in the unsupervised stage of likehood function based on the inspiration of the primate visual cortex analysis. While using the CD (contrastive divergence) algorithm to maximize the objective function, the sparse constraint is used to obtain the sparse distribution of the training set, so that the unlabeled data can be learnt to an intuitive feature representation. Aiming at the problem of invariance existing in sparse regular term, an improved sparse deep belief network is proposed, which uses Laplace distribution to induce the sparse state of hidden layer nodes. At the same time, the position parameter in the distribution is used to control the sparse strength, that is, according to the deviation degree of  the activation probability of the hidden unit from the given sparse value, there are different sparse levels. Through validation analysis on the MNIST and Pendigits handwritten data sets and compared with other existing methods, this method consistently achieves the best recognition accuracy and good sparseness.

Key words: deep belief network (DBN), likelihood function, sparsity constraint, Laplace distribution, location parameter

摘要： 深度置信网络（deep belief network，DBN）通过逐层无监督学习进行训练，但训练过程中易产生大量冗余特征，进而影响特征提取能力。为了使模型更具有解释和辨别能力，基于对灵长类视觉皮层分析的启发，在无监督学习阶段的似然函数中引入惩罚正则项，使用CD（contrastive divergence）训练最大化目标函数的同时，通过稀疏约束获得训练集的稀疏分布，可以使无标签数据学习到直观的特征表示。其次，针对稀疏正则项中存在的不变性问题，提出一种改进的稀疏深度置信网络，使用拉普拉斯函数的分布诱导隐含层节点的稀疏状态，同时将该分布中的位置参数用来控制稀疏的力度，即根据隐藏单元的激活概率与给定稀疏值的偏差程度而具有不同的稀疏水平。通过在MNIST和Pendigits手写体数据集上进行验证分析，并与多种现有方法相比，该方法始终达到最好识别准确度，并且具有良好的稀疏性能。

关键词: 深度置信网络（DBN）, 似然函数, 稀疏约束, 拉普拉斯分布, 位置参数