Few-Shot Image Classification Method with Feature Maps Enhancement Prototype

doi:10.3778/j.issn.1673-9418.2302015

Abstract

Abstract: Due to the scarcity of labeled samples, the class prototype obtained by support set samples is difficult to represent the real distribution of the whole class in metric-based few-shot image classification methods. Meanwhile, samples of the same class may also have large difference in many aspects and the large intra-class bias may make the sample features deviate from the class center. Aiming at the above problems that may seriously affect the performance, a few-shot image classification method with feature maps enhancement prototype (FMEP) is proposed. Firstly, this paper selects some similar features of the query set sample feature maps with cosine similarity and adds them to class prototypes to obtain more representative prototypes. Secondly, this paper aggregates similar features of the query set to alleviate the problem caused by large intra-class bias and makes features distribution of the same class closer. Finally, this paper compares enhanced prototypes and aggregated features which are both closer to real distribution to get better results. The proposed method is tested on four commonly used few-shot classification datasets, namely MiniImageNet, TieredImageNet, CUB-200 and CIFAR-FS. The results show that the proposed method can not only improve the performance of the baseline model, but also obtain better performance compared with the same type of methods.

Key words: few-shot learning, image classification, metric learning, feature maps enhancement prototype, cosine similarity

摘要： 在基于度量学习的小样本图像分类方法中，由于标注样本的稀缺，仅用支持集样本得到的类原型往往难以代表整个类别的真实分布；同时，同类样本间也可能在多个方面存在较大差异，较大的类内差异可能使样本特征偏离类别中心。针对上述可能严重影响图像分类性能的问题，提出一种采用特征图增强原型的小样本图像分类方法（FMEP）。首先，用余弦相似度从查询集样本特征图中选择部分相似特征加入类原型中，得到更具代表性的特征图增强原型；其次，对相似的查询集样本特征进行聚合，缓解类内差异大导致的问题，使同类样本的特征分布更接近；最后，用在特征空间中与真实类别分布都更接近的特征图增强原型和聚合查询特征进行相似度比较得到更优的分类结果。所提方法在MiniImageNet、TieredImageNet、CUB-200和CIFAR-FS等常用的小样本图像分类数据集上进行了实验，结果表明所提方法获得了比基线模型更优的分类性能，同时也优于同类型的小样本图像分类方法。

关键词: 小样本学习, 图像分类, 度量学习, 特征图增强原型, 余弦相似度

XU Huajie, LIANG Shuwei. Few-Shot Image Classification Method with Feature Maps Enhancement Prototype[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(4): 990-1000.

许华杰, 梁书伟. 采用特征图增强原型的小样本图像分类方法[J]. 计算机科学与探索, 2024, 18(4): 990-1000.

References

[1] RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252.
[2] CHEN W Y, LIU Y C, KIRA Z, et al. A closer look at few-shot classification[C]//Proceedings of the 2018 International Conference on Learning Representations, Vancouver, Apr 30-May 3, 2018.
[3] LIU B, CAO Y, LIN Y, et al. Negative margin matters: under-standing margin in few-shot classification[C]//LNCS 12349: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 438-455.
[4] VINYALS O, BLUNDELL C, LILLICRAP T, et al. Matching networks for one shot learning[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Dec 5-12, 2016: 3637-3645.
[5] SNELL J, SWERSKY K, ZEMEL R. Prototypical networks for few-shot learning[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, Dec 4-9, 2017: 4080-4090.
[6] SUNG F, YANG Y, ZHANG L, et al. Learning to compare: relation network for few-shot learning[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Piscataway: IEEE, 2018: 1199-1208.
[7] CHEN Y, LIU Z, XU H, et al. Meta-baseline: exploring simple meta-learning for few-shot learning[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Oct 10-17, 2021. Piscataway: IEEE, 2021: 9042-9051.
[8] 曾武, 毛国君. 多尺度特征聚合的小样本学习方法[J]. 计算机工程与应用, 2023, 59(15): 151-159.
ZENG W, MAO G J. Few-shot learning method for multi-scale feature aggregation[J]. Computer Engineering and Appli-cations, 2023, 59(15): 151-159.
[9] KANG D, KWON H, MIN J, et al. Relational embedding for few-shot classification[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Oct 10-17, 2021. Piscataway: IEEE, 2021: 8802-8813.
[10] XIE J, LONG F, LV J, et al. Joint distribution matters: deep Brownian distance covariance for few-shot classification[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, Jun 18-22, 2022. Piscataway: IEEE, 2022: 7962-7971.
[11] LIU Y, ZHANG W, XIANG C, et al. Learning to affiliate: mutual centralized learning for few-shot classification[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, Jun 18-22, 2022. Piscataway: IEEE, 2022: 14391-14400.
[12] 刘鑫, 周凯锐, 何玉琳, 等. 基于度量的小样本分类方法研究综述[J]. 模式识别与人工智能, 2021, 34(10): 909-923.
LIU X, ZHOU K R, HE Y L, et al. Survey of metric-based few-shot classification[J]. Pattern Recognition and Artificial Intelligence, 2021, 34(10): 909-923.
[13] 安胜彪, 郭昱岐, 白宇, 等. 小样本图像分类研究综述[J]. 计算机科学与探索, 2023, 17(3): 511-532.
AN S B, GUO Y Q, BAI Y, et al. Survey of few-shot image classification research[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(3): 511-532.
[14] REN M, TRIANTAFILLOU E, RAVI S, et al. Meta-learning for semi-supervised few-shot classification[C]//Proceedings of the 2018 International Conference on Learning Representations, Vancouver, Apr 30-May 3, 2018.
[15] HOU R, CHANG H, MA B, et al. Cross attention network for few-shot classification[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, Dec 8-14, 2019: 4003-4014.
[16] LIU J, SONG L, QIN Y. Prototype rectification for few-shot learning[C]//LNCS 12349: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 741-756.
[17] LIU Y, LEE J, PARK M, et al. Learning to propagate labels: transductive propagation network for few-shot learning[C]//Proceedings of the 2019 International Conference on Learning Representations, New Orleans, May 6-9, 2019.
[18] PAN L, LIU W. Transductive graph-attention network for few-shot classification[C]//Proceedings of the 2022 16th IEEE International Conference on Signal Processing, Beijing, Oct 21, 2022. Piscataway: IEEE, 2022: 190-195.
[19] QIAO L, SHI Y, LI J, et al. Transductive episodic-wise adap-tive metric for few-shot learning[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 3603-3612.
[20] GAO Z, WU Y, JIA Y, et al. Curvature generation in curved spaces for few-shot learning[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Oct 10-17, 2021. Piscataway: IEEE, 2021: 8691-8700.
[21] VEILLEUX O, BOUDIAF M, PIANTANIDA P, et al. Realistic evaluation of transductive few-shot learning[C]//Advances?in?Neural?Information?Processing?Systems?34,?Dec?6-14,?2021: 9290-9302.
[22] FAN Q, PEI W, TAI Y W, et al. Self-support few-shot semantic segmentation[C]//LNCS 13679: Proceedings of the 17th European Conference on Computer Vision, Tel Aviv, Oct 23-27, 2022. Cham: Springer, 2022: 701-719.
[23] BERTINETTO L, HENRIQUES J F, TORR P, et al. Meta-learning with differentiable closed-form solvers[C]//Procee-dings of the 2019 International Conference on Learning Representations, New Orleans, May 6-9, 2019.
[24] 李祥霞, 吉晓慧, 李彬. 细粒度图像分类的深度学习方法[J]. 计算机科学与探索, 2021, 15(10): 1830-1842.
LI X X, JI X H, LI B. Deep learning method for fine-grained image categorization[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(10): 1830-1842.
[25] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[26] ZHOU B, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 2921-2929.