Semi-supervised Concept Factorization Algorithm with Local Coordinate Constraint

doi:10.3778/j.issn.1673-9418.2004012

Abstract

Abstract:

Concept factorization (CF) is an effective image representation algorithm, which has been widely used in dimension reduction, feature extraction, data mining and other fields. However, the traditional CF algorithm cannot make use of the limited label information, and fails to guarantee the sparse parts-based representation. Therefore, a novel semi-supervised CF with local coordinate constraint (SLCF) is proposed, which incorporates the local coor-dinates constraint and the limited label information into the CF. Specifically, SLCF enforces the learned coefficients to be sparse by using the local coordinate constraint, and the label constraint matrix can guarantee that the data points sharing the same label are mapped into the same label in the low-dimensional space, so the discriminative ability of different classes is improved. The efficient alternating iterative updating scheme is designed for optimizing SLCF and its convergence is theoretically provided. Numerical experiments on the COIL20, Yale B and MNIST databases demonstrate the effectiveness of proposed SLCF algorithm. Its clustering performance is better than other state-of-the-art algorithms.

Key words: concept factorization (CF), non-negative matrix factorization (NMF), semi-supervised learning, local coordinate constraint

摘要：

概念分解（CF）算法是一种有效的图像表示算法，目前已经广泛应用于维数约简、特征提取、数据挖掘等机器学习领域中。然而，传统CF算法不能利用有效的标签信息，也不能学习数据的稀疏表示。为此，将局部坐标约束和数据有限的标签信息融入到CF模型中，提出了一种带有局部坐标约束的半监督的概念分解（SLCF）算法。SLCF算法利用局部坐标约束学习数据的稀疏性，数据标签约束矩阵能够保证同类标签的数据映射到低维空间中拥有相同的标签，从而提高了不同类间数据的识别能力。利用交替迭代更新方法对SLCF算法的模型进行求解，证明了算法的收敛性。在COIL20、Yale B以及MNIST数据库上的数值实验表明提出的SLCF算法是有效的，其聚类性能优于其他比较的算法。

关键词: 概念分解（CF）, 非负矩阵分解（NMF）, 半监督学习, 局部坐标约束

LI Huirong, ZHANG Lin, ZHAO Pengjun, LI Chao. Semi-supervised Concept Factorization Algorithm with Local Coordinate Constraint[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(2): 379-388.

李会荣, 张林, 赵鹏军, 李超. 带有局部坐标约束的半监督概念分解算法[J]. 计算机科学与探索, 2021, 15(2): 379-388.

References

[1] LEE D D, SEUNG H S. Learning the parts of objects by non-negative matrix factorization[J]. Nature, 1999, 401: 788-791.
[2] LEE D D, SEUNG H S. Algorithms for non-negative matrix factorization[C]//Proceedings of the 2000 International Con-ference on Neural Information Processing Systems, Denver, 2000. Cambridge: MIT Press, 2000: 556-562.
[3] LONG X Z, LU H T, PENG Y, et al. Graph regularized discriminative non-negative matrix factorization for face recognition[J]. Multimedia Tools and Applications, 2014, 72(3): 2679-2699.
[4] XU W, LIU X, GONG Y H. Document clustering based on non-negative matrix factorization[C]//Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Toronto, Jul 28-Aug 1, 2003. New York: ACM, 2003: 267-273.
[5] SHAHNAZ F, BERRY M W, PAUCA V P, et al. Document clustering using nonnegative matrix factorization[J]. Infor-mation Processing and Management, 2006, 42(2): 373-386.
[6] PENG X J, CHEN D, XU D. Hyperplane-based nonnegative matrix factorization with label information[J]. Information Sciences, 2019, 493: 1-19.
[7] XU W, GONG Y H. Document clustering by concept facto-rization[C]//Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, Jul 25-29, 2004. New York: ACM, 2004: 202-209.
[8] CAI D, HE X F, HAN J W. Locally consistent concept fac-torization for document clustering[J]. IEEE Transactions on Knowledge and Data Engineering, 2011, 23(6): 902-913.
[9] LIU H F, YANG Z, YANG J. Local coordinate concept fac-torization factorization for image representation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(6)：1071-1082.
[10] LI H R, ZHANG J S, LIU J M. Graph-regularized CF with local coordinate for image representation[J]. Journal of Visual Communication and Image Representation, 2017, 49: 392-400.
[11] LIU H F, WU Z H, LI X L, et al. Constrained nonnegative matrix factorization for image representation[J]. IEEE Trans-actions on Pattern Analysis and Machine Intelligence, 2012, 34(7): 1299-1311.
[12] HUA W, HE X F. Discriminative concept factorization for data representation[J]. Neurocomputing, 2011, 74: 3800-3807.
[13] BABAEE M, TSOUKALAS S, BABAEE M, et al. Discri-minative nonnegative matrix factorization for dimensionality reduction[J]. Neurocomputing, 2016, 173: 212-223.
[14] LU M, ZHAO X J, ZHANG L, et al. Semi-supervised concept factorization for document clustering[J]. Information Sciences, 2016, 331: 86-98.
[15] LI F, DU L, REN C H. Multiple kernel concept factorization algorithm based on global fusion[J]. Journal of Computer App-lications, 2019, 39(4): 1021-1026.
李飞, 杜亮, 任超宏. 基于全局融合的多核概念分解算法[J]. 计算机应用, 2019, 39(4): 1021-1026.
[16] CHOU Q H, SHI J L, LI Y, et al. Research on the intrusion detection of network based on the graph regularized conce-ption factorization[J]. Journal of Xinjiang University (Natural Science Edition), 2017, 34(2): 200-205.
仇群辉, 史建立, 李岩, 等. 基于图正则化概念分解的网络入侵检测研究[J]. 新疆大学学报(自然科学版), 2017, 34(2): 200-205.
[17] LIU H F, YANG G M, WU Z H, et al. Constrained concept factorization for image representation[J]. IEEE Transactions on Cybernetics, 2014, 44(7): 1214-1224.
[18] LI H R, ZHANG J S, HU J Y, et al. Graph-based discri-minative concept factorization for data representation[J]. Knowledge-Based Systems, 2017, 118: 70-79.
[19] YU K, ZHANG T, GONG Y H. Nonlinear learning using local coordinate coding[C]//Proceedings of the 23rd Annual Conference on Neural Information Processing Systems, Van-couver, Dec 7-10, 2009. Red Hook: Curran Associates, 2009: 2223-2231.
[20] CAI D, HE X F, HAN J W, et al. Graph regularized non- negative matrix factorization for data representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33 (8): 1548-1560.
[21] DEMPSTER A P, LAIRD N M, RUBIN D B. Maximum likelihood from incomplete data via the EM algorithm[J]. Journal of the Royal Statistical Society: Series B (Metho-dological), 1977, 39(1): 1-38.
[22] CHEN Y, ZHANG J M, CAI D, et al. Non-negative local coordinate factorization for image representation[J]. IEEE Transactions on Image Processing, 2013, 22(3): 404-408.
[23] HE Y C, LU H T, HUANG L, et al. Pairwise constrained concept factorization for data representation[J]. Neural Net-works, 2014, 52: 1-17.