Discriminative and Graph Regularized Nonnegative Matrix Factorization with Kernel Method

doi:10.3778/j.issn.1673-9418.2002041

Abstract

Abstract:

Nonnegative matrix factorization (NMF) is a popular technique for dimension reduction,which has been extensively applied in image clustering and other fields.However,NMF is an unsupervised approach,which does not take the label information of the data and capture the inherent geometrical structure of data space.And NMF is a linear method that can't be used when the data are nonlinear.To this end,discriminative and graph regularized non-negative matrix factorization with kernel method is proposed,which uses the available label information,incorporates the graph into the NMF to capture the inherent geometrical structure and uses the kernel method to avoid the nonlinear data, and the result of factorization can effectively improve the clustering effect.Iterative initialization of variants of the NMF is random.A “warm start”strategy is adopted to avoid randomness in the result.Clustering experi-ments on several image datasets verify the effectiveness of the algorithm proposed in this paper compared with the other state-of-the-art methods.

Key words: nonnegative matrix factorization (NMF), semi-supervised clustering, graph regular, kernel method

摘要：

非负矩阵分解（NMF）是一种非常有效的数据降维方法，广泛应用于图像聚类等领域。然而NMF是一种无监督的方法，没有使用数据的标签信息，也不能捕获数据固有的几何结构，并且这是一种线性的方法，不能处理数据是非线性的情况。为此，提出了一种带核方法的判别图正则非负矩阵分解算法。该算法使用了部分有标签数据的标签信息，加入了图正则项来捕获数据的几何结构，使用核方法解决了数据非线性的问题，分解的结果能够有效地提高聚类效果。一般的非负矩阵分解迭代更新的初始化是随机产生的，使用一种“热启动”的策略，减小了结果的随机性。在几种图片数据集上使用该算法进行聚类实验，并与一些先进算法进行了比较，实验结果证明了该算法的有效性。

关键词: 非负矩阵分解（NMF）, 半监督聚类, 图正则, 核方法

LI Xiangli, ZHANG Ying. Discriminative and Graph Regularized Nonnegative Matrix Factorization with Kernel Method[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(11): 1899-1907.

李向利，张颖. 带核方法的判别图正则非负矩阵分解[J]. 计算机科学与探索, 2020, 14(11): 1899-1907.

References

[1] Wold S, Esbensen K, Geladi P. Principal component analysis[J]. Chemometrics and Intelligent Laboratory Systems, 1987, 2(3): 37-52.
[2] Clunis-Ross C W, Riffenburgh R H. Linear discriminant ana-lysis[J]. Pacific Science, 1960, 14: 251-256.
[3] Andrews H, Patterson C. Singular value decomposition (SVD) image coding[J]. IEEE Transactions on Communications, 1976, 24(4): 425-432.
[4] Lee D D, Seung H S. Learning the parts of objects by non-negative matrix factorization[J]. Nature, 1999, 401(6755): 788-791.
[5] Ding C, He X, Simon H D. On the equivalence of nonne-gative matrix factorization and spectral clustering[C]//Pro-ceedings of the 2005 SIAM International Conference on Data Mining, Newport Beach, 2005. Philadelphia: SIAM, 2005: 606-610.
[6] Kuang D, Ding C, Park H. Symmetric nonnegative matrix factorization for graph clustering[C]//Proceedings of the 2012 SIAM International Conference on Data Mining, Anaheim, 2012. Philadelphia: SIAM, 2012: 106-117.
[7] Lee D D, Seung H S. Algorithms for non-negative matrix factorization[C]//Advances in Neural Information Processing Systems 13: Proceedings of the 14th Annual Neural Infor-mation Processing Systems Conference, Denver, 2000. Cam-bridge: MIT Press, 2001: 556-562.
[8] Cai D, He X, Han J, et al. Graph regularized nonnegative matrix factorization for data representation[J]. IEEE Transa-ctions on Pattern Analysis and Machine Intelligence, 2010, 33(8): 1548-1560.
[9] Babaee M, Tsoukalas S, Babaee M, et al. Discriminative non-negative matrix factorization for dimensionality reduction[J]. Neurocomputing, 2016, 173: 212-223.
[10] Scholkopf B, Smola A, Müller K R. Nonlinear component analysis as a kernel eigenvalue problem[J]. Neural Compu-tation, 1998, 10(5): 1299-1319.
[11] Pekalska E, Haasdonk B. Kernel discriminant analysis for positive definite and indefinite kernels[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 31(6): 1017-1032.
[12] Buciu I, Nikolaidis N, Pitas I. Nonnegative matrix factoriza-tion in polynomial feature space[J]. IEEE Transactions on Neural Networks, 2008, 19(6): 1090-1100.
[13] Ding C H Q, Li T, Jordan M I. Convex and semi-nonnegative matrix factorizations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 32(1): 45-55.
[14] Hoyer P O. Non-negative sparse coding[C]//Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Pro-cessing, Martigny, 2002. Piscataway: IEEE, 2002: 557-565.
[15] Zafeiriou S, Tefas A, Buciu I, et al. Exploiting discriminant information in nonnegative matrix factorization with appli-cation to frontal face verification[J]. IEEE Transactions on Neural Networks, 2006, 17(3): 683-695.
[16] Li H, Zhang J, Shi G, et al. Graph-based discriminative non-negative matrix factorization with label information[J]. Neuro-computing, 2017, 266: 91-100.
[17] Wang J J Y, Gao X. Max-min distance nonnegative matrix factorization[J]. Neural Networks, 2015, 61: 75-84.
[18] Jiang S, Li K, Xu Y D R. Relative pairwise relationship constrained non-negative matrix factorization[J]. IEEE Trans-actions on Knowledge and Data Engineering, 2019, 31(8): 1595-1609.
[19] Peng X, Chen D, Xu D. Hyperplane-based nonnegative mat-rix factorization with label information[J]. Information Sci-ences, 2019, 493: 1-19.
[20] Chen W S, Zhao Y, Pan B, et al. Supervised kernel nonne-gative matrix factorization for face recognition[J]. Neuro-computing, 2016, 205: 165-181.
[21] Liu J M, Chen W S, Pan B B, et al. Nonlinear non-negative matrix factorization with fractional power inner-product kernel for face recognition[C]//Proceedings of the 2017 Interna-tional Conference on Security, Pattern Analysis, and Cyber-netics, Shenzhen, 2017. Piscataway: IEEE, 2017: 406-410.
[22] Sun L, Zhao K, Han C Y, et al. Enhancing hyperspectral unmi-xing with two-stage multiplicative update nonnegative matrix factorization[J]. IEEE Access, 2019, 7: 171023-171031.