Kernel-Induced Incomplete Multi-view Clustering

doi:10.3778/j.issn.1673-9418.1912033

Abstract

Abstract:

With the development of technology, data often have multiple forms which come from multiple sources. The multi-view clustering algorithm aims to use the complementary information existing in different sources for clustering. Although the multi-view clustering algorithm has successful applications in many fields, it still faces many challenges, one of which is how to fully mine information between views to reduce the negative impact of missing instances when multi-view data are incomplete. To this end, the algorithm kernel-induced incomplete multi-view clustering (KIMV) is proposed in this paper. On one hand, the kernel method and the non-negative matrix factorization are used to learn an optimal latent feature matrix in the kernel Hilbert space, on the other hand, the performance of the algorithm is improved by adaptive weighting mechanism and Laplace regularization. Experiments on five datasets demonstrate the advantages of KIMV.

Key words: incomplete multi-view, kernel-induced, Laplacian regularization, adaptive view weighting

摘要：

随着技术的发展，数据往往具有来自不同源的多种形式，多视角聚类算法旨在利用不同源中的互补信息进行聚类。虽然目前多视角聚类算法已在各个领域取得较大发展和成功应用，但是多视角聚类算法仍然面临许多重要挑战，其中一个就是当多个视角的样本存在缺失时，如何充分挖掘数据信息以减少缺失样本带来的负面影响。针对此挑战，提出一种基于核诱导的不完整多视角聚类算法（KIMV）。该方法利用核方法和非负矩阵分解技术在核希尔伯特空间中对所有视角学习一个最优的共性矩阵，并通过视角自适应加权机制和图拉普拉斯正则化提高算法性能。在五个多视角数据集上的实验有效验证了KIMV的上述优势。

关键词: 不完整多视角, 核诱导, 拉普拉斯正则化, 自适应视角加权

ZHANG Wei, DENG Zhaohong, WANG Shitong. Kernel-Induced Incomplete Multi-view Clustering[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(2): 284-293.

张炜, 邓赵红, 王士同. 基于核诱导的不完整多视角聚类[J]. 计算机科学与探索, 2021, 15(2): 284-293.

References

[1] CHAO G, SUN S, BI J. A survey on multi-view clustering[J]. arXiv:1712.06246, 2017.
[2] FAN Y, LIANG J, HE R, et al. Robust localized multi-view subspace clustering[J]. arXiv:1705.07777, 2017.
[3] CLEUZIOU G, EXBRAYAT M, MARTIN L, et al. CoFKM: a centralized method for multiple-view clustering[C]//Proceedings of the 9th IEEE International Conference on Data Mining, Miami, Dec 6-9, 2009. Washington: IEEE Computer Society, 2009: 752-757.
[4] LIU X, JI S, GL?NZEL W, et al. Multiview partitioning via tensor methods[J]. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(5): 1056-1069.
[5] CHANG Y S, NIE F, WANG M Y. Multiview feature analysis via structured sparsity and shared subspace discovery[J]. Neural Computation, 2017, 29(7): 1986-2003.
[6] XIA R K, PAN Y, DU L, et al. Robust multi-view spectral clustering via low-rank and sparse decomposition[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, Québec, Jul 27-31, 2014. Menlo Park: AAAI, 2014: 2149-2155.
[7] LIU J, WANG C, GAO J, et al. Multi-view clustering via joint nonnegative matrix factorization[C]//Proceedings of the 13th SIAM International Conference on Data Mining, Austin, May 2-4, 2013. Philadelphia: SIAM, 2013: 252-260.
[8] SHEN B, SI L. Non-negative matrix factorization clustering on multiple manifolds[C]//Proceedings of the 24th AAAI Conference on Artificial Intelligence, Atlanta, Jul 11-15, 2010. Menlo Park: AAAI, 2010: 575-580.
[9] TRIVEDI A, RAI P, DAUMé III H, et al. Multiview clustering with incomplete views[C]//NIPS Workshop, 2010, 224.
[10] GAO H, PENG Y, JIAN S. Incomplete multi-view clustering [C]//Proceedings of the 9th IFIP TC 12 International Conference on Intelligent Information Processing, Melbourne, Nov 18-21, 2016. Berlin, Heidelberg: Springer, 2016: 245-255.
[11] LI S Y, JIANG Y, ZHOU Z H. Partial multi-view clustering[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, Québec, Jul 27-31, 2014. Menlo Park: AAAI, 2014.
[12] ZHAO H D, LIU H F, FU Y. Incomplete multi-modal visual data grouping[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, Jul 9-15, 2016. Menlo Park: AAAI, 2016: 2392-2398.
[13] HU M, CHEN S. Doubly aligned incomplete multi-view clustering[J]. arXiv:1903.02785, 2019.
[14] SHAO W X, HE L F, PHILIP S Y. Multiple incomplete views clustering via weighted nonnegative matrix factorization with [L2,1] regularization[C]//LNCS 9284: Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Porto, Sep 7-11, 2015. Berlin, Heidelberg: Springer, 2015: 318-334.
[15] SHAO W X, HE L F, LU C T, et al. Online multi-view clustering with incomplete views[C]//Proceedings of the 2016 IEEE International Conference on Big Data, Washington, Dec 5-8, 2016. Washington: IEEE Computer Society, 2016: 1012-1017.
[16] WEN J, ZHANG Z, XU Y, et al. Incomplete multi-view clustering via graph regularized matrix factorization[C]//LNCS 11132: Proceedings of the 2018 European Conference on Computer Vision, Munich, Sep 8-14, 2018. Berlin, Heidelberg: Springer, 2018: 593-608.
[17] WEN J, XU Y, LIU H. Incomplete multiview spectral clustering with adaptive graph learning[J]. IEEE Transactions on Cybernetics, 2020, 50(4): 1418-1429.
[18] WANG H, ZONG L, LIU B, et al. Spectral perturbation meets incomplete multi-view data[J]. arXiv:1906.00098, 2019.
[19] Shawe-Taylor J, Cristianini N. Kernel methods for pattern analysis[M]. New York: Cambridge University Press, 2004.
[20] ZHANG D Q, ZHOU Z H, CHEN S C. Adaptive kernel principal component analysis with unsupervised learning of kernels[C]//Proceedings of the 6th IEEE International Conference on Data Mining, Hong Kong, China, Dec 18-22, 2006. Washington: IEEE Computer Society, 2006: 1178-1182.
[21] BELKIN M, NIYOGI P. Laplacian Eigenmaps and spectral techniques for embedding and clustering[C]//Proceedings of the Neural Information Processing Systems: Natural and Synthetic, Vancouver, Dec 3-8, 2001. Cambridge: MIT Press, 2002: 585-591.
[22] SINGH D, ROY D, MOHAN C K. DiP-SVM: distribution preserving kernel support vector machine for big data[J]. IEEE Transactions on Big Data, 2016, 3(1): 79-90.
[23] YANG J, FRANGI A F, YANG J, et al. KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(2): 230-244.
[24] YIN Q Y, WU S, WANG L. Incomplete multi-view clustering via subspace learning[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, Melbourne, Oct 19-23, 2015. New York: ACM, 2015: 383-392.
[25] AN S N, YUN J M, CHOI S. Multiple kernel nonnegative matrix factorization[C]//Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing, Prague, May 22-27, 2011. Piscataway: IEEE, 2011: 1976-1979.
[26] ZHANG D Q, ZHOU Z H, CHEN S C. Non-negative matrix factorization on kernels[C]//LNCS 4099: Proceedings of the 9th Pacific Rim International Conference on Artificial Intelligence, Guilin, Aug 7-11, 2006. Berlin, Heidelberg: Springer, 2006: 404-412.
[27] LEE H, CICHOCKI A, CHOI S. Kernel nonnegative matrix factorization for spectral EEG feature extraction[J]. Neurocomputing, 2009, 72: 3182-3190.
[28] YANG L, JING L, LIU B, et al. Common latent space identification for heterogeneous co-transfer clustering[J]. Neurocomputing, 2017, 269: 29-39.
[29] ZHOU D, BOUSQUET O, LAL T N, et al. Learning with local and global consistency[C]//Proceedings of the Neural Information Processing Systems. Cambridge: MIT Press, 2003: 321-328.
[30] WANG X Z, AN S F. Research on learning weights of fuzzy production rules based on maximum fuzzy entropy[J]. Journal of Computer Research Development, 2006, 43(4): 673-678.
[31] DENG Z H, WANG S T, WU X S, et al. Robust maximum entropy clustering algorithm RMEC and its outlier labeling[J]. Engineering Science, 2004, 6(9): 38-45.
[32] BOYD S, VANDENBERGHE L. Convex optimization[M]. New York: Cambridge University Press, 2004.
[33] CAI X, NIE F P, HUANG H. Multi-view k-means clustering on big data[C]//Proceedings of the 23rd International Joint Conference on Artificial Intelligence, Beijing, Aug 3-9, 2013. Menlo Park: AAAI, 2013: 2598-2604.