重采样和集成学习相结合的文本多标签分类

doi:10.3778/j.issn.1673-9418.2107033

摘要/Abstract

摘要： 医患纠纷类裁判文书的多标签分类是对其进行高效检索和管理的基础，然而，医患纠纷数据集的类别不平衡和标签共生现象直接影响到文书的多标签分类效果。为此，提出了一种重采样和集成学习相结合的文本多标签分类方案。该方案首先提出一种基于标签集合平均稀疏度的样本重采样算法，用于降低标签共生对重采样的影响，从而改善数据集的类别不平衡性；然后，提出一种基于集成学习的多标签分类算法，其基于重采样后的数据集分别训练出多个基分类器，并对各基分类器以一票否决的投票策略进行组合，从而进一步提升分类器的多标签分类效果。实验结果表明，提出的多标签分类方案不仅适用于医患纠纷类裁判文书，而且适用于其他存在类别不平衡和标签共生问题的文本数据集。

关键词: 类别不平衡, 多标签分类, 集成学习, 重采样算法, 标签共生

Abstract: The multi-label classification of medical dispute judgment documents is the basis of efficient retrieval and management, but its effect is affected directly by the class imbalance and label co-occurrence of medical dispute dataset. Therefore, this paper proposes a multi-label classification scheme based on sample resampling and ensemble learning. The scheme includes two parts: in the first part, a resampling algorithm based on the average sparsity of label set is proposed to reduce the impact of label co-occurrence on resampling, so as to improve the class imbalance of dataset; in the second part, a multi-label classification algorithm based on ensemble learning is proposed. It trains multiple base classifiers based on multiple datasets obtained after resampling, and then combines the base classifiers with the voting strategy of one vote veto, so as to further improve the multi-label classification effect of the classifier. Experimental results show that the scheme proposed in this paper is not only suitable for medical dispute judgment documents, but also available for other text datasets with class imbalanced and label co-occurrence problems.

Key words: class imbalance, multi-label classification, ensemble learning, resampling algorithm, label co-occurrence

王天昊, 张沛, 张昭, 陈西海, 王晶, 张柏礼. 重采样和集成学习相结合的文本多标签分类[J]. 计算机科学与探索, 2023, 17(4): 892-901.

WANG Tianhao, ZHANG Pei, ZHANG Zhao, CHEN Xihai, WANG Jing, ZHANG Baili. Multi-label Classification Based on Resampling and Ensemble Learning[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(4): 892-901.

参考文献

[1] ZHANG M L, ZHOU Z H. A review on multi-label learning algorithms[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(8): 1819-1837.
[2] PEREIRA R B, PLASTINO A, ZADROZNY B, et al. Categorizing feature selection methods for multi-label class-ification[J]. Artificial Intelligence Review, 2018, 49(1): 57-78 .
[3] WU Y P, LIN H T. Progressive random k-labelsets for cost-sensitive multi-label classification[J]. Machine Learning, 2017, 106(5): 671-694.
[4] READ J, PFAHRINGER B, HOLMES G, et al. Classifier chains for multi-label classification[J]. Machine Learning, 2011, 85(3): 333-359.
[5] TSOUMAKAS G, KATAKIS I. Multi-label claasification: an overview[J]. International Journal of Data Warehousing and Mining, 2007, 3(3): 1-13.
[6] XU Y, YANG Y, WANG Z, et al. Prediction of acetylation and succinylation in proteins based on multilabel learning RankSVM[J]. Letters in Organic Chemistry, 2019, 15(4): 275-282.
[7] ZHANG M L, ZHOU Z H. ML-KNN: a lazy learning approach to multi-label learning[J]. Pattern Recognition, 2007, 40(7): 2038-2048.
[8] ZHANG M L, ZHOU Z H. Multilabel neural networks with applications to functional genomics and text categorization[J]. IEEE Transactions on Knowledge and Data Engineering, 2006, 18(10): 1338-1351.
[9] NAM J,KIM J, MENCíA E L, et al. Large-scale multi-label text classification-revisiting neural networks[C]//LNCS 8725: Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, Nancy, Sep 15-19, 2014. Cham: Springer, 2014: 437-452.
[10] KURATA G, BING X, ZHOU B W. Improved neural network-based multi-label classification with better initialization leveraging label co-occurrence[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, Jun 12-17, 2016. Stroudsburg: ACL, 2016: 521-526.
[11] CHARTE F, RIVERA A J, DEL JESUS M J, et al. Addressing imbalance in multilabel classification: measures and random resampling algorithms[J]. Neurocomputing, 2015, 163: 3-16.
[12] YAMAN E, SUBASI A. Comparison of Bagging and Boosting ensemble machine learning methods for automated EMG signal classification[J]. BioMed Research International, 2019: 9152506.
[13] CHARTE F, RIVERA A J R, JESUS M J, et al. A first approach to deal with imbalance in multi-label datasets[C]//LNCS 8073: Proceedings of the 8th International Conference on Hybrid Artificial Intelligent Systems, Salamanca, Sep 11-13, 2013. Berlin, Heidelberg: Springer, 2013: 150-160.
[14] TAHIR M A, KITTLER J, BOURIDANE A. Multilabel classification using heterogeneous ensemble of multi-label classifiers[J]. Pattern Recognition Letters, 2012, 33(5): 513-523.
[15] 王昊, 邓三鸿, 苏新宁. 基于字序列标注的中文关键词抽取研究[J]. 现代图书情报技术, 2011(12): 39-45.
WANG H, DENG S H, SU X N. Research on Chinese keyword extraction based on character sequence annotation[J]. Modern Library and Information Technology, 2011(12): 39-45.
[16] 张静. 基于深度学习的中文评论观点抽取研究[D]. 成都: 西南交通大学, 2018.
ZHANG J. Research on opinion extraction of Chinese reviews based on deep learning[D]. Chengdu: Southwest Jiaotong University, 2018.
[17] 肖雨奇. 多标签学习应用于中医诊断帕金森中类别不均衡问题研究[D]. 南京: 南京大学, 2016.
XIAO Y Q. Application of multi label learning in the diagnosis of Parkinson’s disease in traditional Chinese medicine[D]. Nanjing: Nanjing University, 2016.