Multi-view Fuzzy Clustering Combining Visual and Hidden Information with Feature Weighting

doi:10.3778/j.issn.1673-9418.2005015

Abstract

Abstract:

Multi-view clustering is a type of multi-view learning method applied to unsupervised learning, which aims to use the feature set of different views to enhance the effect of clustering. Although multi-view clustering has been effectively applied in many fields, it still faces many challenges. For example, traditional algorithms only use visual information for clustering, ignoring the importance of hidden information. This paper proposes a multi-view fuzzy clustering algorithm (MVSH) that considers both visual and hidden information and feature weighting. This algorithm implements collaborative learning from various views under the framework of fuzzy clustering. On the one hand, personalized information is obtained by clustering features weighted under each view. On the other hand, the coefficient matrix shared by multi-view data sets is extracted by feature learning to obtain common (hidden) information. A multi-view learning with visual and hidden collaboration is realized. Using visual information and hidden information can better balance the common information and personalized information in the process of multi-view clustering collaborative learning. Experimental studies on multi-view data sets also effectively verify the above advantages of MVSH. Comparison with the performance of multiple related algorithms shows that the proposed method can achieve better or at least comparable performance. Experiments on multiple multi-view data sets prove that the strategy of merging visual and hidden information proposed in this paper has better effect than using visual and hidden information alone.

Key words: hidden information, feature weighting, multi-view learning, collaborative learning

摘要：

多视角聚类是一类应用于无监督学习的多视角学习方法，其旨在利用不同视角的特征集去提升聚类的效果。虽然目前多视角聚类已经在很多领域取得了有效的应用，但仍然面临很多挑战，例如传统的算法仅仅利用显性信息进行聚类，忽视了隐性信息的重要性。提出一种兼顾显隐信息与特征加权的多视角模糊聚类算法（MVSH），该算法在模糊聚类框架下实现各视角的协同学习。一方面，通过为每个视角下的特征加权进行聚类，得到个性化信息；另一方面，以特征学习的方式抽取多视角数据集共享的系数矩阵，得到共性（隐）信息，实现了一种显隐视角协同的多视角学习。使用显信息和隐信息可以在多视角聚类协同学习的过程中较好地平衡视角间共性信息和个性化信息。在多视角数据集的实验研究也有效验证了MVSH的上述优点。和多个相关算法的性能比较表明该方法能得到更好或至少相当的性能。通过在多个多视角数据集上的实验证明，提出的融合显隐信息的策略比单独使用显、隐信息时有更好的效果。

关键词: 隐信息, 特征加权, 多视角学习, 协同学习

LIANG Ling, DENG Zhaohong, WANG Shitong. Multi-view Fuzzy Clustering Combining Visual and Hidden Information with Feature Weighting[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(6): 1092-1102.

梁凌, 邓赵红, 王士同. 兼顾显隐信息与特征加权的多视角模糊聚类[J]. 计算机科学与探索, 2021, 15(6): 1092-1102.

References

[1] YU H, WANG X, WANG G, et al. An active three-way clus-tering method via low-rank matrices for multi-view data[J]. Information Sciences, 2020, 507: 823-839.
[2] WANG Y, LIN X, WU L, et al. Robust subspace clustering for multi-view data by exploiting correlation consensus[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3939-3949.
[3] HAN Y, WU F, TAO D, et al. Sparse unsupervised dimensio-nality reduction for multiple view data[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(10): 1485-1496.
[4] XU J, HAN J, NIE F, et al. Re-weighted discriminatively emb-edded k-means for multi-view clustering[J]. IEEE Transactions on Image Processing, 2017, 26(6): 3016-3027.
[5] TANG C, ZHU X, LIU X, et al. Learning a joint affinity graph for multiview subspace clustering[J]. IEEE Transactions on Multimedia, 2018, 21(7): 1724-1736.
[6] PEDRYCZ W. Collaborative fuzzy clustering[J]. Pattern Recog-nition Letters, 2002, 23(14): 1675-1686.
[7] YAMANISHI Y, VERT J P, KANEHISA M. Protein network inference from multiple genomic data: a supervised approach[J]. Bioinformatics, 2004, 20(S1): 363-370.
[8] LU C, YAN S, LIN Z. Convex sparse spectral clustering: single-view to multi-view[J]. IEEE Transactions on Image Process-ing, 2016, 25(6): 2833-2843.
[9] CHEN X, XU X, HUANG J Z, et al. TW-k-means: auto-mated two-level variable weighting clustering algorithm for multiview data[J]. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(4): 932-944.
[10] LIU J, WANG C W, GAO J L, et al. Multi-view clustering via joint nonnegative matrix factorization[C]//Proceedings of the 13th SIAM International Conference on Data Mining, Austin, May 2-4, 2013. Philadelphia: SIAM, 2013: 252-260.
[11] CAO X C, ZHANG C Q, FU H Z, et al. Diversity-induced multi-view subspace clustering[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 586-594.
[12] DENG Z H, CHOI K S, JIANG Y, et al. A survey on soft subspace clustering[J]. Information Sciences, 2016, 348: 84-106.
[13] DENG Z H, LIU R X, ZHANG T, et al. Multi-view clustering with the cooperation of visible and hidden views[J]. IEEE Transactions on Knowledge and Data Engineering, 2020: 1.
[14] DING C H Q, LI T, JORDAN M I. Convex and semi-nonnegative matrix factorizations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 32(1): 45-55.
[15] DIETERICH T G. Ensemble methods in machine learning [C]//LNCS 1857: Proceedings of the 1st International Workshop on Multiple Classifier Systems, Italy, Jun 21-23, 2000. Berlin, Heidelberg: Springer, 2000: 1-15.
[16] LIU Y, YAO X. Ensemble learning via negative correlation[J]. Neural Networks, 1999, 12(10): 1399-1404.
[17] MITRA S, BANKA H, PEDRYCZ W. Rough-fuzzy collaborative clustering[J]. IEEE Transactions on Cybernetics, 2006, 36(4): 795-805.
[18] PEDRYCZ W, RAI P. Collaborative clustering with the use of fuzzy C-means and its quantification[J]. Fuzzy Sets and Systems, 2008, 159(18): 2399-2427.
[19] FORESTIER G, GAN?ARSKI P, WEMMERT C. Collaborative clustering with background knowledge[J]. Data & Knowledge Engineering, 2010, 69(2): 211-228.
[20] GAO H C, NIE F P, LI X L, et al. Multi-view subspace clus-tering[C]//Proceedings of the 2015 IEEE International Con-ference on Computer Vision, Santiago, Dec 7-13, 2015. Wash-ington: IEEE Computer Society, 2015: 4238-4246.
[21] ZHANG C Q, HU Q H, FU H Z, et al. Latent multi-view subspace clustering[C]//Proceedings of the 2017 IEEE Con-ference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 4279-4287.
[22] FAN Y, LIANG J, HE R, et al. Robust localized multi-view subspace clustering[J]. arXiv:1705.07777, 2017.
[23] NANDY M, BANERJEE M. A comparative analysis of application of niblack and sauvola binarization to retinal vessel segmentation[C]//Proceedings of the 3rd International Conference on Computational Intelligence and Networks, Odisha, Oct 28, 2017. Washington: IEEE Computer Society, 2017: 105-109.
[24] ZENG S, WANG X, CUI H, et al. A unified collaborative multikernel fuzzy clustering for multiview data[J]. IEEE Transactions on Fuzzy Systems, 2017, 26(3): 1671-1687.
[25] GUO P, XIE G, LI R. Object detection using multiview CCA-based graph spectral learning[J]. Journal of Circuits, Systems and Computers, 2020, 29(2): 2050022.
[26] LIN T, MA S, ZHANG S. On the global linear convergence of the ADMM with multiblock variables[J]. SIAM Journal on Optimization, 2015, 25(3): 1478-1497.
[27] SEUNG D, LEE L. Algorithms for non-negative matrix fac-torization[J]. Advances in Neural Information Processing Systems, 2001, 13: 556-562.
[28] ZHANG X, ZHANG X, LIU H, et al. Multi-task multi-view clustering[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(12): 3324-3338.
[29] GUO G B, ZHANG J, YORKE-SMITH N. Leveraging multi-views of trust and similarity to enhance clustering-based recommender systems[J]. Knowledge-Based Systems, 2015, 74: 14-27.
[30] ZHAO L, CHEN Z, YANG Y, et al. Incomplete multi-view clustering via deep semantic mapping[J]. Neurocomputing, 2018, 275: 1053-1062.
[31] WANG S T, CHUNG K F L, DENG Z H, et al. Robust maxi-mum entropy clustering algorithm with its labeling for outliers[J]. Soft Computing, 2006, 10(7): 555-563.
[32] GU Q, ZHOU J. Co-clustering on manifolds[C]//Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, Jun 28-Jul 1, 2009. New York: ACM, 2009: 359-368.
[33] HOUTHUYS L, LANGONE R, SUYKENS J A K. Multi-view kernel spectral clustering[J]. Information Fusion, 2018, 44: 46-56.