Journal of Frontiers of Computer Science and Technology ›› 2021, Vol. 15 ›› Issue (3): 553-563.DOI: 10.3778/j.issn.1673-9418.1912005

• Theory and Algorithm • Previous Articles     Next Articles

Incremental Reduced Least Squares Twin Support Vector Regression

CAO Jie, GU Binjie, XIONG Weili, PAN Feng   

  1. Key Laboratory of Advanced Process Control for Light Industry, Ministry of Education, Jiangnan University, Wuxi, Jiangsu 214122, China
  • Online:2021-03-01 Published:2021-03-05

增量式约简最小二乘孪生支持向量回归机

曹杰顾斌杰熊伟丽潘丰   

  1. 江南大学 轻工过程先进控制教育部重点实验室,江苏 无锡 214122

Abstract:

In the incremental least squares twin support vector regression, to solve the problem that the constituted kernel matrix cannot approximate the original kernel matrix well, this paper proposes an incremental reduced least squares twin support vector regression (IRLSTSVR) algorithm. Firstly, in order to reduce the correlation of column vectors in the kernel matrix, the proposed algorithm utilizes a reduced method to determine the correlation between column vectors and then screen support vectors from samples to constitute the column vectors of the kernel matrix. Therefore, the constituted kernel matrix can better approximate the original counterpart, which ensures the sparsity of the solution. Secondly, the inverse matrix is incrementally updated by the block matrix inverse lemma, which further shortens the training time of the proposed algorithm. Finally, the feasibility and efficacy of the proposed algorithm are verified on the benchmark datasets. Experimental results show that the IRLSTSVR algorithm can obtain sparse solution and its generalization performance is closer to offline algorithm compared with state-of-the-art algorithms.

Key words: least squares, twin support vector regression (TSVR), reduced method, incremental learning

摘要:

为了解决增量式最小二乘孪生支持向量回归机存在构成的核矩阵无法很好地逼近原核矩阵的问题,提出了一种增量式约简最小二乘孪生支持向量回归机(IRLSTSVR)算法。该算法首先利用约简方法,判定核矩阵列向量之间的相关性,筛选出用于构成核矩阵列向量的样本作为支持向量以降低核矩阵中列向量的相关性,使得构成的核矩阵能够更好地逼近原核矩阵,保证解的稀疏性。然后通过分块矩阵求逆引理高效增量更新逆矩阵,进一步缩短了算法的训练时间。最后在基准测试数据集上验证算法的可行性和有效性。实验结果表明,与现有的代表性算法相比,IRLSTSVR算法能够获得稀疏解和更接近离线算法的泛化性能。

关键词: 最小二乘, 孪生支持向量回归机(TSVR), 约简方法, 增量式学习