Research of Remote Access Trojan Early Detection Method Using Sequence Analysis

doi:10.3778/j.issn.1673-9418.2007087

Abstract

Abstract:

Remote access Trojan (RAT) is a kind of malware. The main intent of RAT is to steal confidential information and it seriously threatens the security of cyberspace. Most of current network-based RAT detection methods have high requirement on the integrity of the data stream, and their detection are delayed to a certain extent. Based on the analysis of the sequence characteristics of the initial traffic of RAT after the session is established, this paper proposes an RAT early detection method using sequence analysis. The proposed method takes the first TCP stream in the interaction between the RAT??s controlled and control ends as the analysis object, and focuses on the first packet that is sent from the internal host to the external network in the stream and whose transmission layer payload is greater than [α] bytes (called information return packet) as well as several subsequent packets. In the proposed method, three-dimensional features including transmission payload size sequence, transmission byte and time interval are extracted, and a machine learning algorithm is used to construct an efficient early detection model. Experimental results show that this method has the ability to quickly detect RAT, and it can detect RAT traffic with a high accuracy through a small number of data packets in the early stage.

Key words: remote access Trojan (RAT), sequence analysis, early detection, network communication behavior

摘要：

远控木马（RAT）是一类以窃取机密信息为主要目的的恶意程序，严重威胁着网络空间安全。现阶段基于网络的远控木马检测方法大多对数据流的完整性有较高的要求，其检测存在一定程度的滞后。在分析远控木马通信会话建立后初期流量的序列特性的基础上，提出了一种利用序列分析的远控木马早期检测方法。该方法以远控木马被控端和控制端交互中第一条TCP流为分析对象，重点关注流中由内部主机向外部网络发送且数据包传输层负载大于[α]字节的第一个数据包（上线包）及其后续数个数据包，从中提取包含传输负载大小序列、传输字节数和时间间隔在内的三维特征并运用机器学习算法构建了高效的早期检测模型。实验结果表明，该方法具备快速检测远控木马的能力，其通过远控木马会话建立后初期的少量数据包即可高准确率地检测出远控木马流量。

关键词: 远控木马（RAT）, 序列分析, 早期检测, 网络通信行为

WANG Chen, GUO Chun, SHEN Guowei, CUI Yunhe. Research of Remote Access Trojan Early Detection Method Using Sequence Analysis[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(12): 2315-2326.

王晨, 郭春, 申国伟, 崔允贺. 利用序列分析的远控木马早期检测方法研究[J]. 计算机科学与探索, 2021, 15(12): 2315-2326.

References

[1] FARINHOLT B, REZAEIRAD M, PEARCE P, et al. To catch a ratter: monitoring the behavior of amateur DarkComet RAT operators in the wild[C]//Proceedings of the 2017 IEEE Sym-posium on Security and Privacy, San Jose, May 22-26, 2017. Washington: IEEE Computer Society, 2017: 770-787.
[2] 国家互联网应急中心. 2019年中国互联网网络安全报告[EB/OL]. [2020-07-20]. https://www.cert.org.cn/publish/main/ upload/File/2019%20CNCERT%20Cybersecurity%20analysis. pdf.
[3] Proofpoint. Proofpoint Q3 2019 threat report — Emotet??s return, RATs reign supreme, and more[EB/OL]. [2020-07-20]. https://www.proofpoint.com/us/threat-insight/post/proofpoint-q3-2019-threat-report-emotets-return-rats-reign-supreme-and-more.
[4] ZIMBA A, CHEN H S, WANG Z S, et al. Modeling and detection of the multi-stages of advanced persistent threats attacks based on semi-supervised learning and complex networks characteristics[J]. Future Generation Computer Systems, 2020,106: 501-517.
[5] YIN K S, KHIN M A. Network behavioral features for dete-cting remote access Trojans in the early stage[C]//Proceedings of the VI International Conference on Network, Commun-ication and Computing, Kunming, Dec 8-10, 2017. New York: ACM, 2017: 92-96.
[6] YAMADA M, MORINAGA M, UNNO Y, et al. RAT-based malicious activities detection on enterprise internal networks[C]//Proceedings of the 10th International Conference for Internet Technology and Secured Transactions, London, Dec 14-16, 2015. Piscataway: IEEE, 2015: 321-325.
[7] ZHU H Y, QIAO H, WU Z X, et al. A network behavior analysis method to detect reverse remote access Trojan[C]//Proceedings of the 2018 IEEE 9th International Conference on Software Engineering and Service Science, Beijing, Nov 23-25, 2018. Piscataway: IEEE, 2018: 1007-1010.
[8] YIN K S, KHINE M A. Optimal remote access Trojans detection based on network behavior[J]. International Journal of Electrical and Computer Engineering, 2019, 9(3): 2177-2184.
[9] AHMADI M, ULYANOV D, SEMENOV S, et al. Novel feature extraction, selection and fusion for effective malware family classification[C]//Proceedings of the 6th ACM on Conference on Data and Application Security and Privacy, New Orleans, Mar 9-11, 2016. New York: ACM, 2016: 183-194.
[10] RHODE M, BURNAP P, JONES K. Early stage malware prediction using recurrent neural networks[J]. Computers & Security, 2018, 77: 578-594.
[11] CANALI D, LANZI A, BALZAROTTI D, et al. A quantitat-ive study of accuracy in system call-based malware detection[C]//Proceedings of the 2012 International Symposium on Software Testing and Analysis, Minneapolis, Jul 15-20, 2012. New York: ACM, 2012: 122-132.
[12] VIDAL J M, OROZCO A L S, GARCíA-VILLALBA L J. Alert correlation framework for malware detection by anomaly-based packet payload analysis[J]. Journal of Network and Computer Applications, 2017, 97: 11-22.
[13] CHEN H, CHEN J H, XIAO C L, et al. Intrusion detection method of multiple classifiers under deep learning model[J]. Journal of Frontiers of Computer Science and Technology, 2019, 13(7): 1123-1133.
陈虹, 陈建虎, 肖成龙, 等. 深度学习模型下多分类器的入侵检测方法[J]. 计算机科学与探索, 2019, 13(7): 1123-1133.
[14] SANTIKELLUR P, HAQUE T, AL-ZEWAIRI M, et al. Optimized multi-layer hierarchical network intrusion detection system with genetic algorithms[C]//Proceedings of the 2019 2nd International Conference on New Trends in Computing Sciences, Amman, Oct 9-11, 2019. Piscataway: IEEE, 2019: 1-7.
[15] LI W, LI L H, LI J, et al. Characteristics analysis of traffic behavior of remote access Trojan in three communication phases[J]. Netinfo Security, 2015, 15(5): 10-15.
李巍, 李丽辉, 李佳, 等. 远控型木马通信三阶段流量行为特征分析[J]. 信息网络安全, 2015, 15(5): 10-15.
[16] JIANG W, WU X D, CUI X, et al. A highly efficient remote access Trojan detection method[J]. International Journal of Digital Crime and Forensics, 2019, 11(4): 1-13.
[17] JIANG W. A highly efficient remote access Trojan detection method: CN107370752A[P]. 2017-11-21.
姜伟. 一种高效的远控木马检测方法: CN107370752A[P]. 2017-11-21.
[18] JIANG D, OMOTE K. An approach to detect remote access Trojan in the early stage of communication[C]//Proceedings of the 29th IEEE International Conference on Advanced Information Networking and Applications, Gwangju, Mar 24-27, 2015. Washington: IEEE Computer Society, 2015: 706-713.
[19] ADACHI D, OMOTE K. A host-based detection method of remote access Trojan in the early stage[C]//LNCS 10060:Proceedings of the 12th International Conference on Infor-mation Security Practice and Experience, Zhangjiajie, Nov 16-18, 2016. Cham: Springer, 2016: 110-121.
[20] SONG Z H, GUO C, JIANG C H. A fast Trojan detection method based on network traffic analysis[J]. Computer and Modernization, 2019(6): 9-15.
宋紫华, 郭春, 蒋朝惠. 一种基于网络流量分析的快速木马检测方法[J]. 计算机与现代化, 2019(6): 9-15.
[21] WU S, LIU S L, LIN W, et al. Detecting remote access Trojans through external control at area network borders[C]//Proceedings of the 2017 ACM/IEEE Symposium on Archi-tectures for Networking and Communications Systems, Beijing, May 18-19, 2017. Washington: IEEE Computer Society, 2017: 131-141.
[22] BEAUCHESNE N, PRENGER R J. Method and system for detecting external control of compromised hosts: US2015- 0264069[P]. 2015-09-17.
[23] PALLAPROLU S C, NAMAYANJA J M, JANEJA V P, et al. Label propagation in big data to detect remote access Trojans[C]//Proceedings of the 2016 IEEE International Conference on Big Data, Washington, Dec 5-8, 2016. Washington: IEEE Computer Society, 2016: 3539-3547.