Abstract:

Deep semi-supervised learning methods based on consistency regularization and entropy minimization can effectively improve the performance of large-scale neural networks and reduce the need for labeled data. However, the regularization losses of the existing consistency regularization methods do not consider the differences between samples and the negative impact of mislabeled predictions, whereas the entropy minimization methods cannot flexibly adjust the prediction probability distribution. Firstly, to alleviate the negative impact of discrepancies between unlabeled samples and mislabeled predictions, a new consistency loss function named dynamically weighted consistency regularization (DWCR) is proposed, which can dynamically weigh the consistency loss of unlabeled samples. Then, to further adjust the prediction probability distribution, a new loss function called self-confidence promotion loss (SCPL) is proposed, which can flexibly adjust the strength of the model to generate low entropy prediction and achieve low-density separation between classes, thus improving classification performance. Finally, a deep semi-supervised learning method named dynamic consistency and confidence (DCC) is proposed by combining the DWCR, SPCL and supervised loss. Experiments on several datasets show that the proposed method achieves better classification performance than the state-of-the-art deep semi-supervised learning methods.

Key words: deep semi-supervised learning, image classification, dynamic weighted consistency, confident prediction, low density separation

摘要：

基于一致性正则化和熵最小化的深度半监督学习方法可以有效提升大型神经网络的性能，减少对标记数据的需求。然而，现有一致性正则化方法的正则损失没有考虑样本之间的差异及错误预测的负面影响，而熵最小化方法则不能灵活调节预测概率分布。首先，针对样本之间的差别以及错误预测带来的负面影响，提出了新的一致性损失函数，称为动态加权一致性正则化（DWCR），可以实现对无标记数据一致性损失的动态加权。其次，为了进一步调节预测概率分布，提出了新的促进低熵预测的损失函数，称为自信促进损失（SCPL），能灵活调节促进模型输出低熵预测的强度，实现类间的低密度分离，提升模型的分类性能。最后，结合动态加权一致性正则化、自信促进损失与有监督损失，提出了名为动态一致自信（DCC）的深度半监督学习方法。多个数据集上的实验表明，所提出方法的分类性能优于目前较先进的深度半监督学习算法。

关键词: 深度半监督学习, 图像分类, 动态加权一致性, 自信预测, 低密度分离