Denoising Latent Subspace Based Subspace Learning for Image Classification

doi:10.3778/j.issn.1673-9418.2104109

Abstract

Abstract:

To solve the problem that the performance of discriminant least squares regression (DLSR) is not robust to image noise in image classification, a denoising latent subspace based subspace learning (DLSSL) image classification algorithm is proposed. This method is different from the existing classification algorithm based on regression in framework. It introduces a latent subspace in the visual space and label space, and improves the traditional one-step image classification framework to two-step, that is, noise reduction before classification. This method firstly extracts high-order features of data into latent subspace by incomplete autoencoder, then uses the high-order features for regression classification. At the same time, the distance between samples in the class is controlled by the group kernel norm constraint. The introduction of latent subspace brings more flexibility to the algorithm framework, alleviates the differences of dimensions and characteristics between visual space and label space, makes the incomplete autoencoder effective in noise reduction, and improves the robustness of classification algorithm. A number of comparison experiments are designed on the face, biometric, object and deep feature datasets. The experimental results show that the proposed algorithm has strong robustness to the noise in the image, and the obtained projection matrix is more discriminative. Compared with the related image classification algorithms, this algorithm has better performance and stronger universality. Thus it can be effectively applied to various image classification tasks.

Key words: autoencoder, subspace learning, low-rank, denoising, image classification

摘要：

针对判别最小二乘回归（DLSR）对图像噪声鲁棒性不佳的问题，提出一种基于潜子空间去噪的子空间学习图像分类方法（DLSSL）。该方法在架构上不同于现有基于回归的分类方法，其在视觉空间与标签空间中引入一个潜在子空间，将传统的图像分类框架改进为两步，即先降噪后分类。该方法先通过欠完备自编码将数据的高阶特征提取到潜在空间，再利用此高阶特征进行回归分类，同时辅以组核范数约束控制类内样本间距离。潜在子空间的引入为算法框架带来了更多灵活性，缓解了视觉空间与标签空间中数据维度与特性的差异，使得欠完备自编码可以有效地对数据进行降噪，提升了分类算法的鲁棒性。在人脸、生物指纹、物体和深度特征数据集上设计了多组对比实验，实验结果表明，算法对于图像中的噪声具有较强的鲁棒性，获得的投影矩阵具有良好的判别性，相比现有图像分类算法，性能更好、普适性更强，能有效地运用于各种图像分类任务。

关键词: 自编码器, 子空间学习, 低秩, 降噪, 图像分类

YANG Zhangjing, WANG Wenbo, HUANG Pu, ZHANG Fanlong. Denoising Latent Subspace Based Subspace Learning for Image Classification[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(12): 2374-2389.

杨章静, 王文博, 黄璞, 张凡龙. 基于潜子空间去噪的子空间学习图像分类方法[J]. 计算机科学与探索, 2021, 15(12): 2374-2389.

References

[1] WANG B, HUANG Z Q, CHEN L X. Circular features des-cription: effective method for leaf image retrieval and class-ification[J]. Journal of Software, 2019, 30(4): 1148-1163.
王斌, 黄竹芹, 陈良宵. 圆周特征描述: 有效的叶片图像分类和检索方法[J]. 软件学报, 2019, 30(4): 1148-1163.
[2] KONG J, SUN Q S, XU H, et al. Multi-object classification of remote sensing image based on affine-invariant supervised discrete Hashing[J]. Journal of Software, 2019, 30(4): 914-926.
孔颉, 孙权森, 徐晖, 等. 基于仿射不变离散哈希的遥感图像多目标分类[J]. 软件学报, 2019, 30(4): 914-926.
[3] LIU Y, LEI Y B, FAN J L, et al. Survey on image classifica-tion technology based on small sample learning[J]. Acta Auto-matica Sinica, 2021, 47(2): 297-315.
刘颖, 雷研博, 范九伦, 等. 基于小样本学习的图像分类技术综述[J]. 自动化学报, 2021, 47(2): 297-315.
[4] CHEN Z, WU X J, CAI Y H, et al. Sparse non-negative transition subspace learning for image classification[J]. Signal Processing, 2021, 183(8): 107988.
[5] CHEN M?S, HUANG L, WANG C?D, et?al. Multiview sub-space clustering with grouping effect[J]. IEEE Transactions on Cybernetics, 2020. DOI:10.1109/TCYB.2020.3035043.
[6] ZHAO P, WANG C Y, ZHANG S Y, et al. A zero-shot image classification method based on subspace learning with the fusion of reconstruction[J]. Chinese Journal of Computers, 2021, 44(2): 409-421.
赵鹏, 汪纯燕, 张思颖, 等. 一种基于融合重构的子空间学习的零样本图像分类方法[J]. 计算机学报, 2021, 44(2): 409-421.
[7] WAN J, WU F, HE Y B, et al. Clustering algorithm for high-dimensional data under new dimensionality reduction criteria[J]. Journal of Frontiers of Computer Science and Techno-logy, 2020, 14(1): 96-107.
万静, 吴凡, 何云斌, 等. 新的降维标准下的高维数据聚类算法[J]. 计算机科学与探索, 2020, 14(1): 96-107.
[8]?COVER T P?H. Nearest neighbor pattern classification[J]. IEEE Transactions on Circuits and Systems for Video Tech-nology, 2004, 14(1): 4-20.
[9] WRIGHT J, YANG A Y, GANESH A, et al. Robust face reco-gnition via sparse representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 31(2): 210-227.
[10] NASEEM I, TOGNERI R, BENNAMOUN M. Linear reg-ression for face recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(11): 2106-2112.
[11] XU J, AN W, ZHANG L, et al. Sparse, collaborative, or non-negative representation: which helps pattern classification?[J]. Pattern Recognition, 2019, 88: 679-688.
[12] HOERL A?E, KENNARD R W. Ridge regression: biased estimation for nonorthogonal problems[J]. Technometrics, 1970, 12(1): 55-67.
[13] XIANG S M, Nie F P, MENG G F, et al. Discriminative least squares regression for multiclass classification and feature selection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23(11): 1738-1754.
[14] ZHANG X?Y, WANG L, XIANG S, et?al. Retargeted least squares regression algorithm[J]. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(9): 2206-2213.
[15] FANG X Z, TENG S H, LAI Z H, et?al. Robust latent sub-space learning for image classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 29(6): 2502-2515.
[16] CANDE E?J, LI X D, MA Y, et?al. Robust principal com-ponent analysis[J]. Journal of the ACM, 2011, 58(3): 11.
[17] CHEN Z, WU X?J, KITTLER J. Low-rank discriminative least squares regression for image classification[J]. Signal Processing, 2020, 173: 107485.
[18] LU C?S. Solution of the matrix equation AX+XB = C[J]. Electronics Letters, 2007, 7(8): 185-186.
[19] BOYD S, PARIKH N, CHU E, et al. Distributed optimiza-tion and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends in Machine learning, 2011, 3(1): 1-122.
[20] ZHAO J, FENG Q, ZHAO L. Alternating direction and Taylor expansion minimization algorithms for unconstrained nuclear norm optimization[J]. Numerical Algorithms, 2019, 82(1): 371-396.
[21] CHEN C, HE B, YE Y, et?al. The direct extension of ADMM for multi-block convex minimization problems is not neces-sarily convergent[J]. Mathematical Programming, 2016, 155(1/2): 57-79.
[22] LIU G, LIN Z, YAN S, et?al. Robust recovery of subspace structures by low-rank representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 171-184.
[23] ZHANG F L, YANG J, TAI Y, et?al. Double nuclear norm-based matrix decomposition for occluded image recovery and background modeling[J]. IEEE Transactions on Image Processing, 2015, 24(6): 1956-1966.
[24] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409. 1556, 2014.
[25] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recogni-tion, Las Vegas， Jun 27-30, 2016. Washington: IEEE Com-puter Society, 2016: 770-778.
[26] HOWARD A G, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[J]. arXiv:1704.04861, 2017.
[27] CHOLLET F. Xception: deep learning with depthwise separ-able convolutions[C]//Proceedings of the 2017 IEEE Con-ference on Computer Vision and Pattern Recognition, Ho-nolulu,?Jul 21-26, 2017. Washington: IEEE Computer So-ciety, 2017: 1251-1258.
[28] MARTINEZ A, BENAVENTE R. The AR face database[R]. Universitat Autonoma de Barcelona, 1998.
[29] GEORGHIADES A?S, BELHUMEUR P?N, KRIEGMAN D?J. From few to many: illumination cone models for face recognition under variable lighting and pose[J]. IEEE Trans-actions on Pattern Analysis and Machine Intelligence, 2002, 23(6): 643-660.
[30] SIM T, BAKER S, BSAT M. The CMU pose, illumination, and expression database[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 25(12): 1615-1618.
[31] ZHANG D, GUO Z, LU G, et?al. An online system of multi-spectral palmprint verification[J]. IEEE Transactions on Instrumentation and Measurement, 2010, 59(2): 480-490.
[32] YU P?F, ZHOU H, LI H?Y. Personal identification using finger-knuckle-print based on local binary pattern[J]. Applied Mechanics & Materials, 2014, 441: 703-706.
[33] NENE S A. Columbia object image library(COIL-20)[R]. Columbia University, 1996.
[34] RALLINGS C, THRASHER M, GUNTER C, et?al. The FERET database and evaluation procedure for face-recognition algorithms[J]. Image and Vision Computing, 1998, 16(5): 295-306.