关键词: 深度置信网络（DBN）, 似然函数, 稀疏约束, 拉普拉斯分布, 位置参数

Abstract: Deep belief network (DBN) is trained through layer-by-layer unsupervised learning，but it is easy to generate a large amount of redundant information in the training process and affecting the feature extraction ability. In order to make the model more explanatory and discriminative, a penalty regularization term is introduced in the unsupervised stage of likehood function based on the inspiration of the primate visual cortex analysis. While using the CD (contrastive divergence) algorithm to maximize the objective function, the sparse constraint is used to obtain the sparse distribution of the training set, so that the unlabeled data can be learnt to an intuitive feature representation. Aiming at the problem of invariance existing in sparse regular term, an improved sparse deep belief network is proposed, which uses Laplace distribution to induce the sparse state of hidden layer nodes. At the same time, the position parameter in the distribution is used to control the sparse strength, that is, according to the deviation degree of  the activation probability of the hidden unit from the given sparse value, there are different sparse levels. Through validation analysis on the MNIST and Pendigits handwritten data sets and compared with other existing methods, this method consistently achieves the best recognition accuracy and good sparseness.

Key words: deep belief network (DBN), likelihood function, sparsity constraint, Laplace distribution, location parameter