计算机科学与探索

• 学术研究 •    下一篇

网络威胁技战术情报识别提取生成式技术研究

于丰瑞, 杜彦辉   

  1. 1. 中国人民公安大学 信息网络安全学院, 北京  100038
    2. 内蒙古警察职业学院, 呼和浩特  010051

Research on Generative Techniques for Identifying and Extracting Cyber Threat Tactics, Techniques, and Procedures

YU Fengrui,  DU Yanhui   

  1. 1. Institute of Information and Network Security, People’s Public Security University of China, Beijing 100038, China
    2. Inner Mongolia Police Professional College, Hohhot 010051, China

摘要: MITRE ATT&CK定义了网络攻击全过程14类战术625类技术,逐步成为网络威胁技战术情报(Tactics、Techniques and Procedures,TTPs)的事实标准,现有研究基于此分类将TTPs识别提取问题转化为句子级别的战、技术类别多分类任务,利用深度学习、基于提示工程的大语言模型进行问题研究。但限于数据集小样本类别占比大、多分类模型性能瓶颈问题,导致类别识别覆盖率与精度较低。提出结合ChatGPT数据增强和指令监督微调大语言模型的方法,较好的解决了句子级别技术类别多分类问题。ChatGPT数据增强方法在保留原始样本语义基础上更好的丰富了样本多样性,为小样本学习高性能识别提供了高质量训练数据支撑,实验结果也证明了本数据增强方法的优越性;指令监督微调大语言模型,突破了深度学习多分类模型的性能瓶颈,实现625类技术类别识别全覆盖,Precision, Recall和F1值分别达到了86.2%,89.9%和88.0%,优于已有研究。

关键词: 网络威胁情报(CTI), TTPs, ATT&, CK, 数据增强, 大语言模型, 监督微调

Abstract: The MITRE ATT&CK framework defines 14 tactics and 625 techniques that cover the full spectrum of cyber attacks. It has progressively become the de facto standard for describing Tactics, Techniques, and Procedures (TTPs) in cyber threat intelligence. Current research often transforms the task of identifying and extracting TTPs into a multi-class classification problem at the sentence level, employing deep learning and large language models based on prompt engineering. However, issues such as the dominance of small sample categories in datasets and the performance limitations of multi-class models result in low coverage and accuracy in category identification. This paper proposes a method that combines ChatGPT data augmentation with instruction-supervised fine-tuning of large language models, effectively addressing the multi-class classification problem for technique categories at the sentence level. The ChatGPT data augmentation method enriches sample diversity while preserving the original sample semantics, providing high-quality training data to support high-performance recognition in small sample learning. Experimental results demonstrate the superiority of this data augmentation method. The instruction-supervised fine-tuning of the large language model overcomes the performance bottleneck of deep learning multi-class models, achieving full coverage of the 625 technique categories. The Precision, Recall, and F1-score reach 86.2%, 89.9%, and 88.0%, respectively, surpassing existing research.

Key words: cyber threat intelligence(CTI), TTPs, ATT&, CK, data augmentation, large language model, supervised fine-tuning(SFT)