DE-ELM-SSC+：Semi-supervised Classification Algorithm

doi:10.3778/j.issn.1673-9418.1912001

Abstract

Abstract:

The combinations of evolutionary algorithms (EA) and analytical methods have been extensively studied in the fields of machine learning in recent years. This paper focuses on how to combine a differential evolution (DE) algorithm with the semi-supervised classification algorithm based on extreme learning machine (ELM). Firstly, this paper proposes a semi-supervised classification algorithm based on DE and ELM (DE-ELM-SSC) with roughly three steps. Firstly, multiple differential evolution strategies are adopted to optimize the input weights and hidden biases of extreme learning machine, and an optimal strategy for the target data set is selected according to the root mean square error (RMSE). Secondly, the optimal evolutionary strategy selected in the previous step is applied to the DE algorithm to optimize the ELM network parameters. Thirdly, in order to construct a semi-supervised classification model, tri-training technology is used to realize the cooperative training of three improved ELM base classifiers. Then, a nonlinear method is adopted to improve the existing inertial strategy method and realize adaptive adjustment of scaling factor, so as to optimize the DE-ELM-SSC algorithm to obtain the DE-ELM-SSC+ algorithm. Lastly, a large number of experimental results on UCI data sets show that the DE-ELM-SSC+ algorithm outperforms the baseline methods with higher classification accuracy because of evolution strategy selection and improved scaling factor adaptive adjustment.

Key words: extreme learning machine (ELM), semi-supervised classification, strategy selection, differential evolution (DE), scaling factor

摘要：

演化算法和分析方法的结合是机器学习领域近几年的一个研究热点。研究如何将差分进化（DE）演化算法与基于超限学习机（ELM）的半监督分类算法相结合。首先，提出了一种基于DE和ELM的半监督分类方法（DE-ELM-SSC），该算法大致步骤为：采用多种差分进化策略对超限学习机输入权重和偏置参数进行优化，并根据均方根误差选出一个适合目标数据集的最优策略；将上一步选出的最优进化策略应用于DE算法，从而达到优化ELM网络参数的目的；为了构造半监督分类预测模型，采用Tri-training技术实现了三个改进ELM基分类器的协同训练。然后，采用非线性方法改进现有惯性策略方法，实现了缩放因子自适应调整，从而优化了DE-ELM-SSC算法，得到DE-ELM-SSC+算法。UCI标准数据集上的大量实验结果表明，DE-ELM-SSC+算法能根据数据集选择合适的进化策略，并自适应调整缩放因子，获得比Baseline方法更高的分类准确率。

关键词: 超限学习机, 半监督分类, 策略选择, 差分进化, 缩放因子

PANG Jun, HUANG Heng, ZHANG Shou, SHU Zhiliang, ZHAO Yuhai. DE-ELM-SSC+：Semi-supervised Classification Algorithm[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(12): 2014-2027.

庞俊，黄恒，张寿，舒智梁，赵宇海. DE-ELM-SSC+半监督分类算法[J]. 计算机科学与探索, 2020, 14(12): 2014-2027.

References

[1] Qian C, Shi J C, Tang K, et al. Constrained monotone k-sub-modular function maximization using multiobjective evolu-tionary algorithms with theoretical guarantee[J]. IEEE Tran-sactions on Evolutionary Computation, 2017, 22(4): 595-608.
[2] Zhou Z H, Yu Y, Qian C. Evolutionary learning: advances in theories and algorithms[M]. Berlin, Heidelberg: Springer, 2019.
[3] Yu L, Hu L, Tang L. Stock selection with a novel sigmoid-based mixed discrete-continuous differential evolution algo-rithm[J]. IEEE Transactions on Knowledge and Data Engin-eering, 2016, 28(7): 1891-1904.
[4] Wang H, Hu W, Qiu Z, et al. Nodes?? evolution diversity and link prediction in social networks[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(10): 2263-2274.
[5] Sun Y, Yen G G, Yi Z. Evolving unsupervised deep neural networks for learning meaningful representations[J]. IEEE Transactions on Evolutionary Computation, 2018, 23(1): 89-103.
[6] Chen Y, Lin Y, Hu X M. Parallel differential evolution with multi-population and multi-strategy[J]. Journal of Frontiers of Computer Science and Technology, 2014, 8(12): 1502-1510.陈颖, 林盈, 胡晓敏. 多种群多策略的并行差分进化算法[J]. 计算机科学与探索, 2014, 8(12): 1502-1510.
[7] Sabar N R, Abawajy J, Yearwood J. Heterogeneous cooper-ative co-evolution memetic differential evolution algorithm for big data optimization problems[J]. IEEE Transactions on Evolutionary Computation, 2016, 21(2): 315-327.
[8] Storn R, Price K. Differential evolution—a simple and effi-cient adaptive scheme for global optimization over con-tinuous spaces[M]. Berkeley: ICSI, 1995: 15-20.
[9] Mezura-Montes E, Velázquez-Reyes J, Coello C A C. A com-parative study of differential evolution variants for global optimization[C]//Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, Seattle, Jul 8-12, 2006. New York: ACM, 2006: 485-492.
[10] Wu M S, Deng X G. Semi-supervised pattern classification method based on Tri-DE-ELM[J]. Computer Engineering and Applications, 2018, 54(3): 109-114.吴明胜, 邓晓刚. 基于Tri-DE-ELM的半监督模式分类方法研究[J]. 计算机工程与应用, 2018, 54(3): 109-114.
[11] Zhou Z H, Li M. Tri-training: exploiting unlabeled data using three classifiers[J]. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(11): 1529-1541.
[12] Yuan J G, Sun Z G, Qu G J. Simulation study of differential evolution[J]. Journal of System Simulation, 2007, 19(20):4646-4648.袁俊刚, 孙治国, 曲广吉. 差异演化算法的数值模拟研究[J]. 系统仿真学报, 2007, 19(20): 4646-4648.
[13] Jiang L Q, Guo Z, Liu G B. Study on the strategy of scaling factor in differential evolution algorithm[C]//Proceedings of the 1st Advanced Instrumentation, Automation and Integration Technology Conference, Chongqing/Lijiang, Dec 7, 2007: 508-510.姜立强, 郭铮, 刘光斌. 差分进化算法缩放因子取值策略研究[C]//2007年首届仪表、自动化与先进集成技术大会, 重庆·云南丽江, 2007: 508-510.
[14] Price K, Storn R M, Lampinen J A. Differential evolution: a practical approach to global optimization[M]. Berlin, Hei-delberg: Springer, 2006.
[15] Ye Q Q, Yang L H, Fu Y G, et al. Classification approach based on improved belief rule-base reasoning[J]. Journal of Frontiers of Computer Science and Technology, 2016, 10(5): 709-721.叶青青, 杨隆浩, 傅仰耿, 等. 基于改进置信规则库推理的分类方法[J]. 计算机科学与探索, 2016, 10(5): 709-721.
[16] Wang Y, Xu B, Sun G, et al. A two-phase differential evo-lution for uniform designs in constrained experimental dom-ains[J]. IEEE Transactions on Evolutionary Computation, 2017, 21(5): 665-680.
[17] Heredia J P. Modelling evolutionary algorithms with sto-chastic differential equations[J]. Evolutionary Computation, 2018, 26(4): 657-686.
[18] Liu C, Lin Y, Hu X M. Analyses and comparisons of dif-ferent update strategies for differential evolution[J]. Journal of Frontiers of Computer Science and Technology, 2013, 7(11): 983-993.刘琛, 林盈, 胡晓敏. 差分演化算法各种更新策略的对比分析[J]. 计算机科学与探索, 2013, 7(11): 983-993.
[19] Sharma P, Sharma H, Kumar S, et al. A review on scale factor strategies in differential evolution algorithm[M]//Soft Computing for Problem Solving. Singapore: Springer, 2019: 925-943.
[20] Huang G B, Zhu Q Y, Siew C K. Extreme learning mach-ine: theory and applications[J]. Neurocomputing, 2006, 70: 489-501.
[21] Pang J, Gu Y, Xu J, et al. Parallel multi-graph classification using extreme learning machine and MapReduce[J]. Neuro-computing, 2017, 261: 171-183.
[22] Liu T, Lekamalage C K L, Huang G B, et al. Extreme lear-ning machine for joint embedding and clustering[J]. Neuro-computing, 2018, 277: 78-88.
[23] Liu J, Chen Y, Liu M, et al. SELM: semi-supervised ELM with application in sparse calibrated location estimation[J]. Neurocomputing, 2011, 74(16): 2566-2572.
[24] Huang G, Song S, Gupta J N D, et al. Semi-supervised and unsupervised extreme learning machines[J]. IEEE Transac-tions on Cybernetics, 2014, 44(12): 2405-2417.
[25] Zhou Y, Liu B, Xia S, et al. Semi-supervised extreme lear-ning machine with manifold and pairwise constraints regu-larization[J]. Neurocomputing, 2015, 149: 180-186.
[26] Pei H, Wang K, Lin Q, et al. Robust semi-supervised extr-eme learning machine[J]. Knowledge-Based Systems, 2018, 159: 203-220.
[27] Xie J, Liu S, Dai H. Manifold regularization based distributed semi-supervised learning algorithm using extreme learning machine over time-varying network[J]. Neurocomputing, 2019, 355: 24-34.
[28] Li K, Zhang J, Xu H, et al. A semi-supervised extreme learning machine method based on co-training[J]. Journal of Computational Information Systems, 2013, 9(1): 207-214.
[29] Wu Y L. Research on graph-based semi-supervised learning[D]. Hefei: University of Science and Technology of China, 2008.吴毓龙. 基于图的半监督学习的研究[D]. 合肥: 中国科学技术大学, 2008.
[30] Liu B, Yang Y H, Zhao Z B, et al. A batch inheritance ext-reme learning machine algorithm based on regular optimi-zation[J]. Journal of Electronics and Information Technology, 2020, 42(7): 1734-1742.刘彬, 杨有恒, 赵志彪, 等. 一种基于正则优化的批次继承极限学习机算法[J]. 电子与信息学报, 2020, 42(7): 1734-1742.
[31] Meng H Y, Zhang X H, Liu S Y. A differential evolution based on double populations for constrained multi-objective optimization problem[J]. Chinese Journal of Computers, 2008, 31(2): 228-235.孟红云, 张小华, 刘三阳. 用于约束多目标优化问题的双群体差分进化算法[J]. 计算机学报, 2008, 31(2): 228-235.
[32] Eiben A E, Hinterding R, Michalewicz Z. Parameter control in evolutionary algorithms[J]. IEEE Transactions on Evolu-tionary Computation, 1999, 3(2): 646-657.
[33] Brest J, Greiner S, Boskovic B, et al. Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems[J]. IEEE Transactions on Evolutionary Computation, 2006, 10(6): 646-657.