Abstract: To enhance classification accuracy in class-incremental learning (CIL) models for text classification and mitigate the issue of catastrophic forgetting, this paper introduces a CIL framework based on a large language model (CIL-LLM). The CIL-LLM framework selects representative samples through sampling and compression, leveraging the strong contextual learning abilities of the LLM to distill key skills, which serve as the basis for classification, thereby reducing storage costs. Keywords matching is used to select optimal skills, which are then formulated into prompts that guide downstream weak LLMs in classification, improving accuracy. Through skill fusion based on knowledge distillation, the framework effectively expands and updates the skill repository while ensuring the learning of both new and old categories. Comparative experiments on the THUCNews dataset show that the CIL-LLM framework improves the average accuracy by 6.3% and reduces performance degradation by 3.1% compared to the existing L-SCL method. In ablation studies, the SLEICL model enhanced by the CIL-LLM framework increases average accuracy by 10.4% and reduces performance degradation by 3.3% across all tasks. These results further validate that sample compression, keyword matching, and skill fusion each contribute to optimizing the accuracy and reducing performance degradation in the model.

Key words: Class-Incremental Learning, Large Language Model (LLM), Thematic Classification, Knowledge Distillation

摘要： 在文本分类领域，为了提升类别增量学习模型的分类准确率并避免灾难性遗忘问题，本文提出了一种基于大语言模型（Large Language Model, LLM）的类别增量学习框架（Class-Incremental Learning of Large Language Model, CIL-LLM）。CIL-LLM框架通过抽样和压缩环节选取具有代表性的样本，利用较强语言理解能力的LLM基于上下文学习提炼关键技能，并这些技能作为分类的依据，从而降低了存储成本；采用关键词匹配环节选取最优技能，以此构建提示词，引导下游弱LLM进行分类，提高了分类的准确性；根据基于知识蒸馏的技能融合环节，不仅实现了技能库的有效拓展和更新，还兼顾了新旧类别特性的学习。对比实验结果表明，在THUCNews数据集上的测试中，与现有的L-SCL方法相比，CIL-LLM框架在所有任务上的平均准确率提升了6.3%，性能下降率降低了3.1%。此外，在消融实验中，经由CIL-LLM框架增强的SLEICL模型相比于原有模型，所有任务的平均准确率提高了10.4%，性能下降率降低了3.3%。消融实验进一步验证了本文提出的样本压缩、关键词匹配和技能融合环节均对模型的准确率和性能下降率产生了优化效果。

关键词: 类别增量学习, 大语言模型(LLM), 主题分类, 知识蒸馏