Style Regularized Least Squares Support Vector Machine Based on Multiple Kernel Learning

doi:10.3778/j.issn.1673-9418.1906018

Abstract

Abstract:

Though current multiple kernel learning algorithms integrate the abilities of different kernel functions on the representation of the physical features of data, they do not make full use of the style information existing in the stylistic dataset. Therefore, a style regularized least squares support vector machine based on multiple kernel learning (MK-SRLSSVM) is proposed. Its basic idea is to consider the style transformation matrices which represent the style information of samples and it is regularized in the objective function. The commonly-used alternating optimization technique is utilized to optimize the objective function. Style transformation matrices and classifier parameters are simultaneously updated during iteration. In order to use the trained style information in the prediction process, two new rules are considered in the traditional prediction model. The unknown patterns are normalized by the learned style transform matrix before being classified. The proposed classifier not only keeps the advantages of existing multi-kernel learning algorithms in representing the physical features of samples, but also exploits the style information contained in the stylistic dataset so as to improve the classification performance effectively. The experimental results on the stylistic datasets confirm the effectiveness of the proposed classifier.

Key words: least squares support vector machines (LSSVM), multiple kernel learning, stylistic data, style information

摘要：

当前的多核学习方法结合了不同核函数在对数据的物理特性表示上的能力，但在风格化数据集中不能充分利用样本中所隐含的风格信息。由此，提出应用于风格化数据的基于多核学习的风格正则化最小二乘支持向量机（MK-SRLSSVM）。算法利用风格转换矩阵表示包含在样本中的风格信息，并在目标函数中对其进行正则化处理，通过常用的交替优化方法对目标函数进行优化，在迭代过程中同步更新风格转换矩阵和分类器参数。为在预测过程中利用已学习的风格信息，在传统预测方法中增加了两种新的规则，在分类之前预先使用风格转换矩阵对样本风格进行标准化处理。所提出的分类器不仅利用了现有多核学习算法在表示样本的物理特征方面的优势，同时有效挖掘了数据集内包含的风格信息以提高分类性能，在风格化数据集中的实验结果证明了算法的有效性。

关键词: 最小二乘支持向量机（LSSVM）, 多核学习, 风格化数据, 风格信息

SHEN Hao, WANG Shitong. Style Regularized Least Squares Support Vector Machine Based on Multiple Kernel Learning[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(9): 1532-1544.

沈浩，王士同. 基于多核学习的风格正则化最小二乘支持向量机[J]. 计算机科学与探索, 2020, 14(9): 1532-1544.

References

[1] Vapnik V N. The nature of statistical learning theory[M]. New York: Springer-Verlag, 1995.
[2] Tao J W, Wang S T. Kernel support vector machine for domain adaptation[J]. Acta Automatica Sinica, 2012, 38(5):797-811.陶剑文, 王士同. 领域适应核支持向量机[J]. 自动化学报, 2012, 38(5): 797-811.
[3] Feng C, Liao S Z. Large-scale kernel methods via random hypothesis spaces[J]. Journal of Frontiers of Computer Science and Technology, 2018, 12(5): 785-793.冯昌, 廖士中. 大规模核方法的随机假设空间方法[J]. 计算机科学与探索, 2018, 12(5): 785-793.
[4] Sch?lkopf B, Mika S, Smola A, et al. Kernel PCA pattern reconstruction via approximation preimages[C]//Proceedings of the 1998 International Conference on Artificial Neural Net-works, Sk?vde, Sep 2-4, 1998. Berlin, Heidelberg: Springer, 1998: 147-152.
[5] Lanckriet G R G, De Bie T, Cristianini N, et al. A stat-istical framework for genomic data fusion[J]. Bioinforma-tics, 2004, 20(16): 2626-2635.
[6] Rakotomamonjy A, Bach F R, Canu S, et al. More efficiency in multiple kernel learning[C]//Proceedings of the 24th Inter-national Conference on Machine Learning, Corvalis, Jun 20-24, 2007. New York: ACM: 775-782.
[7] Li X, Wang S T. Combination of original information and mapping information for multiple kernel learning for dimensionality reduction[J]. Journal of Frontiers of Com-puter Science and Technology, 2019, 13(2): 310-321.李旭, 王士同. 原信息与映射信息组合的多核学习降维方法[J]. 计算机科学与探索, 2019, 13(2): 310-321.
[8] Tao J W, Chung F L, Wang S T, et al. Sparse label propa-gation: a robust domain adaptation learning method[J]. Journal of Software, 2015, 26(5): 977-1000.陶剑文, Chung F L, 王士同, 等. 稀疏标签传播:一种鲁棒的领域适应学习方法[J]. 软件学报, 2015, 26(5): 977-1000.
[9] Serhat S, Rong J, Anil K. Multiple kernel learning for visual object recognition: a review[J]. IEEE Transactions on Pat-tern Analysis and Machine Intelligence, 2014, 36(7): 1354-1369.
[10] Lu M, Liu L H, Wu L H. Research on multi-kernel support vector data description method of classification[J]. Computer Engineering and Applications, 2016, 52(18): 68-73.卢明, 刘黎辉, 吴亮红. 多核支持向量数据描述分类方法研究[J]. 计算机工程与应用, 2016, 52(18): 68-73.
[11] Wang H Q, Sun F C, Cai Y N, et al. On multiple kernel learning methods[J]. Acta Automatica Sinica, 2010, 36(8):1037-1050.汪洪桥, 孙富春, 蔡艳宁, 等. 多核学习方法[J]. 自动化学报, 2010, 36(8): 1037-1050.
[12] Sriharsha V, George N. Style context with second-order statistics[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2005, 27(1): 14.
[13] Cheema M S, Eweiwi A, Bauckhage C. Human activity recognition by separating style and content[J]. Pattern Recog-nition Letters, 2014, 50: 130-138.
[14] Zhang X Y, Huang K Z, Liu C L. Pattern field classification with style normalized transformation[C]//Proceedings of the 22nd International Joint Conference on Artificial Intelli-gence, Catalonia, Jul 16-22, 2011. Menlo Park: AAAI, 2011: 1621-1626.
[15] Jiang H C, Huang K Z, Zhang R. Field support vector regression[C]//LNCS 10634: Proceedings of the 24th Inter-national Conference on Neural Information Processing, Guang-zhou, Nov 14-18, 2017. Berlin, Heidelberg: Springer, 2017: 699-708.
[16] Huang K Z, Jiang H C, Zhang X Y. Field support vector machines[J]. IEEE Transactions on Emerging Topics in Com-putational Intelligence, 2017, 1(6): 454-463.
[17] Pattarin F, Paterlini S, Minerva T. Clustering financial time series: an application to mutual funds style analysis[J].Com-putational Statistics and Data Analysis, 2004, 47(2): 353-372.
[18] Marzinotto G, Rosales J C, El-Yacoubi M A, et al. Age and gender characterization through a two layer clustering of online handwriting[C]//LNCS 9386: Proceedings of the 16th International Conference on Advanced Concepts for Intelligent Vision Systems, Catania, Oct 26-29, 2015. Berlin, Heidelberg: Springer, 2015: 428-439.
[19] Suykens J A K. Least squares support vector machines[J]. International Journal of Circuit Theory & Applications, 2002, 27(6): 605-615.
[20] Chen X K, Guo N, Ma Y D, et al. More efficient sparse multi-kernel based least square support vector machine[J]. Communications in Computer and Information Science, 2012, 289: 70-78.
[21] Kloft M. [lp]-norm multiple kernel learning[J]. IEEE Trans-actions on Pattern Analysis & Machine Intelligence, 2007, 30(2): 348-353.
[22] Manic V, Babu B R. More generality in efficient multiple kernel learning[C]//Proceedings of the 26th Annual International Conference on Machine Learning, Quebec, Jun 14-18, 2009.New York: ACM, 2009: 1065-1072.
[23] Cortes C, Mohri M, Rostamizadeh A. Learning non-linear combinations of kernels[C]//Proceedings of the 23rd Annual Conference on Neural Information Processing Systems, Vancouver, Dec 7-10, 2009. Red Hook: Curran Associates,2009: 396-404.
[24] Mehmet G, Alpaydin E. Localized multiple kernel learning[C]//Proceedings of the 25th International Conference on Machine Learning, Helsinki, Jun 5-9, 2008. New York: ACM, 2008: 352-359.
[25] Cristianini N, Shawe-Taylor J. An introduction to support vector machines and other kernel-based learning methods [M]. Cambridge: Cambridge University Press, 2000.
[26] Xu Z L, Jin R, Yang H Q, et al. Simple and efficient multiple kernel learning by group lasso[C]//Proceedings of the 27th International Conference on Machine Learning, Haifa, Jun 21-24, 2010. Madison: Omni Press, 2010: 1175-1182.
[27] Cortes C, Mohri M, Rostamizadeh A. Two-stage learning kernel algorithms[C]//Proceedings of the 27th International Conference on Machine Learning, Haifa, Jun 21-24, 2010. Madison: Omni Press, 2010: 239-246.
[28] Rakotomamonjy A, Bach F R, Canu S, et al. Simple MKL[J]. The Journal of Machine Learning Research, 2008, 9(11): 2491-2521.
[29] Aiolli F, Donini M. EasyMKL: a scalable multiple kernel learning algorithm[J]. Neurocomputing, 2015, 169: 215-224.
[30] Wall M E, Rechtsteiner A, Rocha L M. Singular value decomposition and principal component analysis[J]. arXiv:physics/0208101, 2003.
[31] Jiang Y, Deng Z H, Chung F L, et al. Recognition of epileptic EEG signals using a novel multiview TSK fuzzy system[J]. IEEE Transactions on Fuzzy Systems, 2017, 25(1): 3-20.