Abstract: Continuous learning aims at continuously learn multiple tasks, and can use the knowledge of previous tasks to help the learning of current task without catastrophic forgetting. Progressive neural network is a parameter-independent continuous learning method, gradually assigning extra networks to each task to improve the performance of continuous learning. However, this method can not directly take advantage of similarity influence between tasks. In the continuous learning process, by comparing the similarity between tasks, the performance of current task may be significantly improved by using this to trim and migrate the parameters of previous task. Therefore, a task-similarity guided progressive deep neural network (TSGPNN) and its learning method are proposed, which include two stages: task-similarity evaluation and progressive learning. In the task-similarity evaluation stage, a reference quantity is defined to measure the similarity between target task domains, and served as a reference for the knowledge transfer between tasks. The progressive process improves the ability to learn new tasks by absorbing the knowledge from previous tasks and relearning. After performing task segmentation on CIFAR-100, MNIST-Permutation and MNIST-Rotation datasets, experimental results show that TSGPNN performance is better and more stable than one-task learning, multi-task learning, and other continuous learning methods.

Key words: catastrophic forgetting, continual learning, deep network, progressive neural network

摘要： 持续学习旨在连续地学习多个任务，且在不发生灾难性遗忘的情况下，能够利用先前任务的知识帮助当前任务的学习。渐进神经网络是一种参数独立的持续学习方法，渐进地为每个任务分配额外的网络来提升持续学习的性能，但是这种方法未能直接利用任务间相似度的影响。而在持续学习过程中，通过对比任务间的相似度，并以此对先前任务的参数进行修剪再迁移可能会显著提高当前任务的性能。因此，提出了一种任务相似度引导的渐进深度神经网络（TSGPNN）及其学习方法，它包括了任务相似度评估和渐进学习两个阶段。其中，任务相似度评估阶段定义了一个参照值来衡量目标任务域之间的相似度，并以此作为任务间知识迁移量的参照；渐进过程通过吸收先前任务中的知识重新学习，以此提升学习新任务的能力。对CIFAR-100、MNIST-Permutation和MNIST-Rotation数据集做任务切分，实验表明，TSGPNN的性能与单任务学习、多任务学习和其他基准持续学习方法相比更好、更稳定。

关键词: 灾难性遗忘, 持续学习, 深度网络, 渐进神经网络