Correlation-Based Method for Tracing Multi-dimensional Time Series Data Anomalies

doi:10.3778/j.issn.1673-9418.2008100

Abstract

Abstract:

This paper proposes a multi-dimensional time series anomaly data detection method based on correlation analysis, to trace the cause of anomaly detection: system failure data and sensor quality problem data are classified, and then real system failures are identified to avoid false detection. Firstly, the time series correlation graph model is proposed, which is further summarized as the time series correlation loop model. The time series correlation set is obtained by extracting the features in the time series correlation cycle, the cause of abnormality is detected, and the system failure is judged according to the result. Through a large number of experiments on real industrial data sets, the effectiveness of the method in the detection of abnormal sources of high-dimensional time series data is verified. Through comparative experiments, it is verified that the method is superior to fundamental algorithms based on statistics and machine learning models in terms of stability and efficiency. The higher dimensionality of time series, the more obvious improvement of the method compared with the fundamental algorithms. This method not only saves the cost, but also realizes the accurate identification of multi-dimensional abnormal data.

Key words: multi-dimensional time series, abnormal detection, correlation analysis, graph algorithm, industrial big data, provenance

摘要：

提出一种基于统计学相关性分析的多维时序异常数据检测分析方法，以对检测中表现为异常的数据进行溯源：对反映系统故障的数据和传感器质量问题的数据进行分类，进而识别出真正的系统故障，避免误检。首先根据相关关系构建时序相关图，再进一步归纳为时序相关环模型，通过在时序相关图上搜索并确定时序相关环的过程，提取时序相关环中的特征，得到时间序列相关性集合。进而利用时间序列相关性集合进行时序数据异常来源检测，根据检测结果评估时序传感器数据对应的系统故障的几率。在真实的工业设备传感器序列数据集上进行大量实验，实验结果验证了该方法在高维时序数据的异常检测任务上的有效性。通过对比实验，验证了该方法从稳定性和效率上优于基于统计和基于机器学习模型的基准算法，时间序列的维度越高，该方法较基准算法的提升越明显。该方法通过对多维时序数据相关性知识的挖掘，既节约了计算成本，又实现了对多维异常数据来源的精准识别。

关键词: 多维时间序列, 异常检测, 相关性分析, 图算法, 工业大数据, 溯源

WANG Muxian, DING Xiaoou, WANG Hongzhi, LI Jianzhong. Correlation-Based Method for Tracing Multi-dimensional Time Series Data Anomalies[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(11): 2142-2150.

王沐贤，丁小欧，王宏志，李建中. 基于相关性的多维时序数据异常溯源方法[J]. 计算机科学与探索, 2021, 15(11): 2142-2150.

References

[1] HAO S, LI G L, FENG J H, et al. Survey of structured data cleaning methods[J]. Journal of Tsinghua University (Science and Technology), 2018, 58(12): 1037-1050.
郝爽, 李国良, 冯建华, 等. 结构化数据清洗技术综述[J]. 清华大学学报(自然科学版), 2018, 58(12): 1037-1050.
[2] GUPTA M, GAO J, AGGARWAL C A, et al. Outlier detec-tion for temporal data: a survey[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(9): 2250-2267.
[3] CHALAMALLA A, ILYAS I F, OUZZANI M, et al. Descri-ptive and prescriptive data cleaning[C]//Proceedings of the 2014 International Conference on Management of Data, Snow-bird, Jun 22-27, 2014. New York: ACM, 2014: 445-456.
[4] KUMAR V. Parallel and distributed computing for cyber-security[J]. IEEE Distributed Systems Online, 2005, 6(10): 1.
[5] GOLDBERGER A L, AMARAL L A N, GLASS L, et al. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals[J]. Circulation, 2000, 101(23): 215-220.
[6] DE STEFANO C, SANSONE C, VENTO M. To reject or not to reject: that is the question-an answer in case of neural classifiers[J]. IEEE Transactions on Systems, Management and Cybernetics, 2000, 30(1): 84-94.
[7] SCH?OLKOPF B, PLATT J C, SHAWE-TAYLOR J C, et al. Estimating the support of a high-dimensional distribution[J]. Neural Computation, 2001, 13(7): 1443-1471.
[8] ROTH V. Kernel Fisher discriminants for outlier detection[J]. Neural Computation, 2006, 18(4): 942-960.
[9] ESTER M, KRIEGEL H P, SANDER J, et al. A density-based algorithm for discovering clusters in large spatial databases with noise[C]//Proceedings of the 2nd International Con-ference on Knowledge Discovery and Data Mining, Portland, 1996. Menlo Park: AAAI, 1996: 226-231.
[10] SMITH R, BIVENS A, EMBRECHTS M, et al. Clustering approaches for anomaly based intrusion detection[C]//Pro-ceedings of the Intelligent Engineering Systems through Ar-tificial Neural Networks, St Louis, 2002. New York: ASME Press, 2002: 579-584.
[11] WANG X, WANG C. Time series data cleaning: a survey[J]. IEEE Access, 2020, 8: 1866-1881.
[12] HARTIGAN J A, WONG M A. A k-means clustering algo-rithm[J]. Applied Statistics, 1979, 28: 100-108.
[13] FUJIMAKI R, NAKATA T, TSUKAHARA H, et al. Mining abnormal patterns from heterogeneous time-series with irre-levant features for fault event detection[J]. Statistical Anal-ysis and Data Mining, 2009, 2(1): 1-17.
[14] QIAO Y, XIN X, BIN Y, et al. Anomaly intrusion detection method based on HMM[J]. Electronics Letters, 2002, 38:663-664.
[15] SONG S X, ZHANG A Q, WANG J M. SCREEN: stream data cleaning under speed constraints[C]//Proceedings of the 2015 ACM SIGMOD International Conference on Man-agement of Data, Melbourne, May 31-Jun 4, 2015. New York: ACM, 2015: 827-841.
[16] CHANDOLA V, BANERJEE A, KUMAR V. Anomaly de-tection: a survey[J]. ACM Computing Surveys, 2009, 41(3): 1-58.
[17] DING X O, YU S J, WANG M X, et al. Anomaly detection on industrial time series based on correlation analysis[J]. Journal of Software, 2020, 31(3): 726-747.
丁小欧, 于晟健, 王沐贤, 等. 基于相关性分析的工业时序数据异常检测[J]. 软件学报, 2020, 31(3): 726-747.
[18] CHAUDHURI P. A self-stabilizing algorithm for detecting fundamental cycles in a graph[J]. Journal of Computer and System Sciences, 1999, 59(1): 84-93.