计算机科学与探索 ›› 2021, Vol. 15 ›› Issue (7): 1265-1278.DOI: 10.3778/j.issn.1673-9418.2005028

• 人工智能 • 上一篇    下一篇

随机特征映射的四层神经网络及其增量学习

杨悦,王士同   

  1. 江南大学 人工智能与计算机学院,江苏 无锡 214122
  • 出版日期:2021-07-01 发布日期:2021-07-09

Novel Four-Layer Neural Network and Its Incremental Learning Based on Randomly Mapped Features

YANG Yue, WANG Shitong   

  1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu 214122, China
  • Online:2021-07-01 Published:2021-07-09

摘要:

提出了一种基于随机特征映射的四层神经网络(FRMFNN)及其增量学习算法。FRMFNN首先把原始输入特征通过特定的随机映射算法转化为随机映射特征存储于第一层隐藏层节点中,再经过激活函数对随机映射特征进行非线性转化生成第二层隐藏节点,最后将第二层隐藏层通过输出权重连接到输出层。由于第一层和第二层隐藏层的权重是根据任意连续采样分布概率随机生成的而不需要训练更新,且输出层的权重可以用岭回归算法快速求解,从而避免了传统反向传播神经网络耗时的训练过程。当FRMFNN没有达到期望精度时,借助于快速的增量算法可以持续改进网络性能,从而避免了重新训练整个网络。详细介绍了FRMFNN及其增量算法的结构原理,证明了FRMFNN的通用逼近性。与宽度学习(BLS)和极限学习机(ELM)的增量学习算法相比,在多个主流分类和回归数据集上的实验结果表明了FRMFNN及其增量学习算法的有效性。

关键词: 神经网络, 机器学习, 随机特征映射, 宽度学习, 通用逼近, 增量学习, 岭回归, 正则化

Abstract:

This paper proposes a four-layer neural network based on randomly feature mapping (FRMFNN) and its fast incremental learning algorithms. First, FRMFNN transforms the original input features into randomly mapped features by certain randomly mapping algorithm and stores them in its nodes of first hidden layer. Then, the FRMFNN generates its nodes of second hidden layer using non-linear activation function on all random mapping features. Finally, the second hidden layer is linked to the output layer through the output weights. Since the weights of the first and the second hidden layers are randomly generated according to certain continuous sampling probability distribution, without the updates of the weights, and the output weights can be quickly solved by the ridge regression, avoiding time-consuming training process of the traditional back propagation neural networks. When FRMFNN can??t reach the prescribed accuracy, its performance can be continuously improved by its rapid incremental algorithm, thereby avoiding retraining the whole network. This paper, a detail introduction of proposed FRMFNN and its incremental algorithms is provided. What??s more, a proof of universal approximation property of FRMFNN is also given. Compared with broad learning system (BLS) and the incremental learning of extreme learning machine (ELM), the experimental results on several popular classification and regression datasets demonstrate the effectiveness of the proposed FRMFNN and its incremental learning algorithms.

Key words: neural network, machine learning, randomly mapped feature, broad learning, universal approximation, incremental learning, ridge regression, regularization