Multivariate Time Series Prediction Model Based on Mixed Features of Time Domain and Frequency Domain

doi:10.3778/j.issn.1673-9418.2410089

Abstract

Abstract: Currently, multivariate time series prediction methods mainly transform time series into frequency domain repre-sentations for feature extraction. But this leads to the loss of some time domain information and accuracy, and the traditional attention mechanism brings quadratic time complexity. To address these issues, a multivariate time series prediction model (TFMformer) based on hybrid time-frequency domain features is proposed. TFMformer uses a multi-scale segmentation operation to decompose more accurate semantics from a multi-scale perspective, enhancing the model??s ability to capture comprehensive semantic information of the series. It reduces the number of input tokens through slicing to lower time complexity. A hybrid time-frequency domain feature enhancement module is introduced to make time domain and frequency domain features fuse and interact, improving overall feature representation. Additionally, time domain feature information is incorporated into frequency domain attention to enhance the frequency domain??s perception of time domain information, enabling the model to focus more precisely on meaningful feature combinations and reducing prediction bias due to the lack of time domain information. TFMformer is tested on six benchmark datasets. Compared with existing advanced methods, the mean squared error and mean absolute error of the prediction results are decreased by an average of 3.8% and 2.8% respectively, and the maximum reduction in mean absolute error reaches 11.2%, which proves the effectiveness of model.

Key words: multivariate time series prediction, Transformer, sequence decomposition, frequency domain attention mechanism, deep learning

摘要： 目前多变量时间序列预测方法主要是将时间序列转换为频域表示来提取特征信息，然而频域表示下序列会损失部分时域信息造成精度损失，且传统注意力机制会产生平方级时间复杂度。针对上述问题，提出了一种基于时域频域混合特征的多变量时序预测模型（TFMformer）。模型采用多尺度切分操作，以多尺度混合的视角分解出更准确语义，增强模型捕获序列综合语义信息的能力并通过切片操作减少模型输入token数来降低时间复杂度。提出时域频域混合特征增强模块，将时域和频域特征进行融合与交互，从而提升整体特征表征能力。同时提出在频域注意力的基础上引入时域特征信息，提升频域空间对时域信息感知能力，使得模型能够更精准地聚焦于有意义的特征组合，减少因时域信息缺失造成的预测偏差。TFMformer模型在6个基准数据集上进行了实验，预测结果与现有的先进方法相比，均方误差和平均绝对误差分别平均下降了3.8%和2.8%，其中平均绝对误差最高下降了11.2%，证明了该模型的有效性。

关键词: 多变量时间序列预测, Transformer, 序列分解, 频域注意力机制, 深度学习

MIN Feng, LIU Yuzhuo, LIU Yuhui, LIU Biao. Multivariate Time Series Prediction Model Based on Mixed Features of Time Domain and Frequency Domain[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(8): 2099-2109.

闵锋, 刘宇卓, 刘煜晖, 刘彪. 基于时域频域混合特征的多变量时序预测模型[J]. 计算机科学与探索, 2025, 19(8): 2099-2109.

References

[1] CAO D, WANG Y, DUAN J, et al. Spectral temporal graph neural network for multivariate time-series forecasting[C]//Advances in Neural Information Processing Systems 33, 2020: 17766-17778.
[2] LI S, JIN X, XUAN Y, et al. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems, 2019: 5243-5253.
[3] 王慧强, 陈楚皓, 吕宏武, 等. 基于双向稀疏Transformer的多变量时序分类模型[J]. 小型微型计算机系统, 2024, 45(3): 555-561.
WANG H Q, CHEN C H, LYU H W, et al. Multivariate time series classification model based on bidirectional sparse transformer[J]. Journal of Chinese Computer Systems, 2024, 45(3): 555-561.
[4] LU J, YAO J, ZHANG J, et al. Soft: softmax-free transformer with linear complexity[C]//Advances in Neural Information Processing Systems 34, 2021: 21297-21309.
[5] ZHOU T, MA Z, WEN Q, et al. FEDformer: frequency enhanced decomposed transformer for long-term series forecasting[C]//Proceedings of the 39th International Conference on Machine Learning, 2022: 27268-27286.
[6] ZHANG Y, YAN J. Crossformer: transformer utilizing cross-dimension dependency for multivariate time series forecasting[C]//Proceedings of the 11th International Conference on Learning Representations, 2023.
[7] 秦音, 郭杜杜, 周飞, 等. 零担物流时序预测的SARIMA-GRU-BPNN组合模型及应用[J]. 计算机工程与应用, 2024, 60(19): 297-308.
QIN Y, GUO D D, ZHOU F, et al. Combined SARIMA-GRU-BPNN model for LTL logistics time series prediction and application[J]. Computer Engineering and Applications, 2024, 60(19): 297-308.
[8] 李欢欢, 黄添强, 丁雪梅, 等. 基于多尺度时空图卷积网络的交通出行需求预测[J]. 计算机应用, 2024, 44(7): 2065-2072.
LI H H, HUANG T Q, DING X M, et al. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network[J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[9] 屈景怡, 杨柳, 陈旭阳, 等. 基于时空序列的Conv-LSTM航班延误预测模型[J]. 计算机应用, 2022, 42(10): 3275-3282.
QU J Y, YANG L, CHEN X Y, et al. Flight delay prediction model based on Conv-LSTM with spatiotemporal sequence[J]. Journal of Computer Applications, 2022, 42(10): 3275-3282.
[10] 戴宇睿, 安俊秀, 陶全桧. 融合双通路注意力与VT-LSTM的金融时序预测[J]. 计算机工程与应用, 2023, 59(12): 157-165.
DAI Y R, AN J X, TAO Q H. Financial time-series prediction by fusing dual-pathway attention with VT-LSTM[J]. Computer Engineering and Applications, 2023, 59(12): 157-165.
[11] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30, 2017: 5998-6008.
[12] 梁宏涛, 刘硕, 杜军威, 等. 深度学习应用于时序预测研究综述[J]. 计算机科学与探索, 2023, 17(6): 1285-1300.
LIANG H T, LIU S, DU J W, et al. Review of deep learning applied to time series prediction[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(6): 1285-1300.
[13] JIANG S, SYED T, ZHU X, et al. Bridging self-attention and time series decomposition for periodic forecasting[C]//Proceedings of the 31st ACM International Conference on Information & Knowledge Management. New York: ACM, 2022: 3202-3211.
[14] 陈东洋, 毛力. 融合增量学习与Transformer模型的股价预测研究[J]. 计算机科学与探索, 2024, 18(7): 1889-1899.
CHEN D Y, MAO L. Research on stock price prediction integrating incremental learning and transformer model[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(7): 1889-1899.
[15] KITAEV N, KAISER L, LEVSKAYA A. Reformer: the efficient transformer[C]//Proceedings of the 8th International Conference on Learning Representations, 2020.
[16] JIANG M W, ZENG P Y, WANG K, et al. FECAM: frequency enhanced channel attention mechanism for time series forecasting[J]. Advanced Engineering Informatics, 2023, 58: 102158.
[17] QUAN J L, ZHAN W F, CHEN Y H, et al. Time series decomposition of remotely sensed land surface temperature and investigation of trends and seasonal variations in surface urban heat islands[J]. Journal of Geophysical Research: Atmospheres, 2016, 121(6): 2638-2657.
[18] SHUMWAY R H, STOFFER D S, STOFFER D S. Time series analysis and its applications[M]. New York: Springer, 2000.
[19] HAMILTON J D. Time series analysis[M]. Princeton: Princeton University Press, 2020.
[20] WU H X, XU J H, WANG J M, et al. Autoformer: decomposition transformers with auto-correlation for long-term series forecasting[EB/OL]. [2024-08-19]. https://arxiv.org/abs/2106.13008.
[21] UCI Machine Learning Repository. Electricity load diagrams 2011-2014[EB/OL]. [2024-08-19]. https://doi.org/10.24432/C58C86.
[22] CHOLLET F. Deep learning with Python[M]. 2nd ed. Greenwich: Manning Publications, 2021.
[23] LAI G, CHANG W C, YANG Y, et al. Modeling long-and short-term temporal patterns with deep neural networks[C]//Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval.New York: ACM, 2018: 95-104.
[24] RIDHO LUBIS A, PRAYUDANI S, AL-KHOWARIZMI. Optimization of MSE accuracy value measurement applying false alarm rate in forecasting on fuzzy time series based on percentage change[C]//Proceedings of the 2020 8th International Conference on Cyber and IT Service Management. Piscataway: IEEE, 2020: 1-5.
[25] SINGH M, DUVAL Q, ALWALA K V, et al. The effectiveness of MAE pre-pretraining for billion-scale pretraining[C]//Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2023: 5461-5471.
[26] NIE Y, NGUYEN N H, SINTHONG P, et al. A time series is worth 64 words: long-term forecasting with transformers[C]//Proceedings of the 11th International Conference on Learning Representations, 2023.
[27] WU Z H, PAN S R, LONG G D, et al. Connecting the dots: multivariate time series forecasting with graph neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2020: 753-763.
[28] WU H, HU T, LIU Y, et al. TimesNet: temporal 2D-variation modeling for general time series analysis[C]//Proceedings of the 11th International Conference on Learning Representations, 2023.
[29] ZENG A L, CHEN M X, ZHANG L, et al. Are transformers effective for time series forecasting?[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(9): 11121-11128.
[30] LAI G K, CHANG W C, YANG Y M, et al. Modeling long- and short-term temporal patterns with deep neural networks[C]//Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. New York: ACM, 2018: 95-104.