Class Incremental Learning Method Integrating Balance Weight and Self-supervision

doi:10.3778/j.issn.1673-9418.2212055

Abstract

Abstract: In view of the catastrophic forgetting phenomenon of knowledge in class incremental learning in image classification, the existing class incremental learning methods focus on the correction of the unbalanced offset of the model classification layer, ignoring the offset of the model feature layer, and fail to solve the problem of the imbalance between the new and old samples faced by class incremental learning. Therefore, a new class incremental learning method is proposed, which is called balance weight and self-supervision (BWSS). BWSS designs an adaptive balance weight based on the low expectation of the old class in training, so as to expand the loss return proportion of the old class in the same data batch to correct the overall model offset. Then, BWSS introduces self-supervised learning to predict the rotation angle of the sample as an auxiliary task, so as to make the model have the expression ability of redundant features and common features to better support incremental tasks. Through the experimental comparison with the mainstream incremental class learning algorithms on the open datasets CIFAR-10 and CIFAR-100, it is proven that BWSS not only has better incremental performance on CIFAR-10 with fewer categories and more samples, but also has advantages on CIFAR-100 with more categories and fewer samples. Ablation experiments and feature visualization demonstrate that the proposed method is effective for the feature representation and incremental performance of the model. The final accuracy of BWSS’s 5-stage incremental task on CIFAR-10 reaches 76.9%, which is 5 percentage points higher than the baseline method.

Key words: incremental learning, self-supervision, problem of imbalance, distillation of knowledge

摘要： 针对图像分类中类增量学习的知识灾难性遗忘现象，现有类增量学习方法着重于模型分类层的不平衡偏移修正，忽视了模型特征层的偏移，未能良好解决类增量学习所面临的新旧类别不平衡问题。为此，提出一种新的类增量学习方法，称为融合平衡权重和自监督的类增量学习方法（BWSS）。首先，BWSS利用旧类在训练中预测期望低的特点设计了自适应的平衡权重，扩大旧类的损失回传占比以修正整个模型的偏移。其次，BWSS引入自监督模块，将样例的旋转角度预测作为辅助任务，强化模型对冗余特征和共性特征的表达能力，以更好地支撑增量任务。通过与主流类增量学习算法在公开数据集CIFAR-10和CIFAR-100上的实验对比，证明BWSS不仅在类别少样例多的CIFAR-10上增量性能更优，在类别多样本少的CIFAR-100上同样具有优势。通过消融实验和特征可视化，验证了所提方法对模型的特征表示能力和增量性能是有效的。BWSS在CIFAR-10上的5阶段增量任务最终准确率达到了76.9%，相比基线方法提高了5个百分点。

关键词: 增量学习, 自监督, 不平衡问题, 知识蒸馏

GONG Jiayi, XU Xinlei, XIAO Ting, WANG Zhe. Class Incremental Learning Method Integrating Balance Weight and Self-supervision[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(2): 477-485.

巩佳义, 许鑫磊, 肖婷, 王喆. 融合平衡权重和自监督的类增量学习方法[J]. 计算机科学与探索, 2024, 18(2): 477-485.

References

[1] 刘冰瑶, 刘进锋. 增量学习研究综述[J]. 现代计算机, 2022, 28(13): 72-75.
LIU B Y, LIU J F. A survey of incremental learning[J]. Modern Computer, 2022, 28(13): 72-75.
[2] LI Z Z, HOIEM D. Learning without forgetting[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(12): 2935-2947.
[3] HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[J]. arXiv:1503.02531, 2015.
[4] RANNEN A, ALJUNDI R, BLASCHKO M, et al. Encoder based lifelong learning[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Piscataway: IEEE, 2017: 1320-1328.
[5] KIRKPATRICK J, PASANU R, RABINOWITZ N, et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the National Academy of Sciences, 2017, 114(13): 3521-3526.
[6] SHIN H, LEE J K, KIM J, et al. Continual learning with deep generative replay[C]//Advances?in?Neural?Information Processing Systems 30,?Long Beach, Dec 4-9, 2017: 2994-3003.
[7] WU C S, HERRANZ L, LIU X, et al. Memory replay GANS: learning to generate new categories without forgetting[C]//Advances in Neural Information Processing Systems 31, Montréal, Dec 3-8,?2018: 5962-5972.
[8] REBUFFI S A, KOLESNIKOV A, SPERL G, et al. iCaRL: incremental classifier and representation learning[C]//Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Piscataway: IEEE, 2017: 5533-5542.
[9] HOU S H, PAN X Y, LOY C C, et al. Learning a unified classifier incrementally via rebalancing[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 831-839.
[10] WU Y, CHEN Y P, WANG L J, et al. Large scale incremental learning[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 374-382.
[11] ZHAO B W, XIAO X, GAN G J, et al. Maintaining discrimination and fairness in class incremental learning[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 13205-13214.
[12] ALJUNDI R, BABILONI F, ELHOSEINY M, et al. Memory aware synapses: learning what (not) to forget[C]//Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 139-154.
[13] DOUILLARD A, CORD M, OLLION C, et al. PODNet: pooled outputs distillation for small-tasks incremental learning[C]//Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Sprin-ger, 2020: 86-102.
[14] ZHU F, ZHANG X Y, WANG C, et al. Prototype augmentation and self-supervision for incremental learning[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 5871-5880.
[15] WANG L X, XU S C, WANG X, et al. Addressing class imbalance in federated learning[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, Feb 2-9, 2021. Menlo Park: AAAI, 2021: 10165-10173.
[16] LEE H, HWANG S J, SHIN J W. Self-supervised label augmentation via input transformations[C]//Proceedings of the 37th International Conference on Machine Learning, Jul 13-18, 2020: 5714-5724.
[17] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Piscataway: IEEE, 2016: 770-778.