时空交叉注意力特征融合的交通流量预测模型

doi:10.3778/j.issn.1673-9418.2412086

摘要/Abstract

摘要： 交通流量预测是智能交通系统的核心组成部分，对高效的交通管理和规划至关重要。针对现有方法在动态时空依赖建模和特征表示方面存在的不足，提出了一种时空交叉注意力特征融合的交通流量预测模型。通过构建动态多特征嵌入模块，融合原始数据、周期性、空间和时空自适应嵌入，生成交通流量数据内在的时空特征表示，提升了模型对多样化交通模式的适应能力。基于Transformer编码器架构，设计并行的时空自注意力模块，高效提取时间和空间特征，为深度的特征融合提供了基础。创新性地引入时空交叉注意力特征融合机制，在时间和空间维度分别使用多头交叉注意力机制，使时间特征能够自适应地学习关键节点的空间信息，同时空间特征也能选择性地聚焦于重要的时间信息，以实现时间和空间特征的深度融合，从而更全面地理解和捕捉交通流量中的动态时空依赖关系。在四个真实交通数据集上的实验结果表明，与最优基线模型相比，所提模型的MAE、RMSE和MAPE指标分别平均降低了1.56%、1.91%和2.58%。

关键词: 交通流量预测, Transformer, 交叉注意力, 特征嵌入

Abstract: Traffic flow prediction is a core component of intelligent transportation systems and is crucial for efficient traffic management and planning. To address the shortcomings of existing methods in dynamic spatio-temporal dependency modeling and feature representation, this paper proposes a traffic flow prediction model based on spatio-temporal cross-attention feature fusion. Firstly, a dynamic multi-feature embedding module is constructed to integrate raw data, periodicity, spatial and spatio-temporal adaptive embeddings, generating intrinsic spatio-temporal feature representations of traffic flow data. This enhances the model??s adaptability to diverse traffic patterns. Secondly, based on the Transformer encoder architecture, a parallel spatio-temporal self-attention module is designed to efficiently extract temporal and spatial features, laying foundation for deep feature fusion. Finally, a spatio-temporal cross-attention feature fusion mechanism is innovatively introduced, employing a multi-head cross-attention mechanism in the temporal and spatial dimensions. This allows temporal features to adaptively learn spatial information from key nodes, while spatial features selectively focus on important temporal information, enabling deep fusion of temporal and spatial features. Consequently, the model achieves a more comprehensive understanding of dynamic spatio-temporal dependencies in traffic flow. Experiments on four real-world traffic datasets demonstrate that, compared with the best baseline model, the proposed model achieves average reductions of 1.56%, 1.91% and 2.58% in MAE, RMSE and MAPE.

Key words: traffic flow prediction, Transformer, cross-attention, feature embedding

孟祥福, 徐永杰, 翁雪. 时空交叉注意力特征融合的交通流量预测模型[J]. 计算机科学与探索, 2025, 19(7): 1931-1944.

MENG Xiangfu, XU Yongjie, WENG Xue. Spatio-Temporal Cross-Attention Feature Fusion Model for Traffic Flow Prediction[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(7): 1931-1944.

参考文献

[1] GARG T, KAUR G. A systematic review on intelligent transport systems[J]. Journal of Computational and Cognitive Engineering, 2023, 2(3): 175-188.
[2] JAVED A R, AHMED W, PANDYA S, et al. A survey of explainable artificial intelligence for smart cities[J]. Electronics, 2023, 12(4): 1020.
[3] HUO G Y, ZHANG Y, WANG B Y, et al. Hierarchical spatio- temporal graph convolutional networks and transformer network for traffic flow forecasting[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(4): 3855-3867.
[4] NIGAM N, SINGH D P, CHOUDHARY J. A review of different components of the intelligent traffic management system (ITMS)[J]. Symmetry, 2023, 15(3): 583.
[5] SATTARZADEH A R, KUTADINATA R J, PATHIRANA P N, et al. A novel hybrid deep learning model with ARIMA Conv-LSTM networks and shuffle attention layer for short-term traffic flow prediction[J]. Transportmetrica A: Transport Science, 2025, 21(1).
[6] XU X C, JIN X F, XIAO D Q, et al. A hybrid autoregressive fractionally integrated moving average and nonlinear autoregressive neural network model for short-term traffic flow prediction[J]. Journal of Intelligent Transportation Systems, 2023, 27(1): 1-18.
[7] DAI G N, KONG W Y, LIU Y B, et al. Multi-perspective convolutional neural networks for citywide crowd flow prediction[J]. Applied Intelligence, 2023, 53(8): 8994-9008.
[8] MO J Q, GONG Z G, CHEN J Y. Attentive differential convolutional neural networks for crowd flow prediction[J]. Knowledge-Based Systems, 2022, 258: 110006.
[9] KHAN A, FOUDA M M, DO D T, et al. Short-term traffic prediction using deep learning long short-term memory: taxonomy, applications, challenges, and future trends[J]. IEEE Access, 2023, 11: 94371-94391.
[10] ZHU J J, JIANG Q S, SHEN Y H, et al. Application of recurrent neural network to mechanical fault diagnosis: a review[J]. Journal of Mechanical Science and Technology, 2022, 36(2): 527-542.
[11] JIANG W W, LUO J Y, HE M, et al. Graph neural network for traffic forecasting: the research progress[J]. ISPRS International Journal of Geo-Information, 2023, 12(3): 100.
[12] JIN G Y, LIANG Y X, FANG Y C, et al. Spatio-temporal graph neural networks for predictive learning in urban computing: a survey[J]. IEEE Transactions on Knowledge and Data Engineering, 2024, 36(10): 5388-5408.
[13] YU B, YIN H T, ZHU Z X. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018: 3634-3640.
[14] LI Y G, YU R, SHAHABI C, et al. Diffusion convolutional recurrent neural network: data-driven traffic forecasting[EB/OL]. [2024-10-23]. https://arxiv.org/abs/1707.01926.
[15] ZHENG C P, FAN X L, WANG C, et al. GMAN: a graph multi-attention network for traffic prediction[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(1): 1234-1241.
[16] JIANG J W, HAN C K, ZHAO W X, et al. PDFormer: propagation delay-aware dynamic long-range transformer for traffic flow prediction[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(4): 4365-4373.
[17] WILLIAMS B M, HOEL L A. Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: theoretical basis and empirical results[J]. Journal of Transportation Engineering, 2003, 129(6): 664-672.
[18] ZIVOT E, WANG J. Vector autoregressive models for multi-variate time series[M]//Modeling financial time series with S-PLUS?. New York: Springer, 2006: 385-429.
[19] XU H B, JIANG C S. Deep belief network-based support vector regression method for traffic flow forecasting[J]. Neural Computing and Applications, 2020, 32(7): 2027-2036.
[20] LIN G C, LIN A J, GU D L. Using support vector regression and K-nearest neighbors for short-term traffic flow prediction based on maximal information coefficient[J]. Information Sciences, 2022, 608: 517-531.
[21] TEDJOPURNOMO D A, BAO Z F, ZHENG B H, et al. A survey on modern deep neural network for traffic prediction: trends, methods and challenges (extended abstract)[C]//Proceedings of the 2023 IEEE 39th International Conference on Data Engineering. Piscataway: IEEE, 2023: 3795- 3796.
[22] SAYED S A, ABDEL-HAMID Y, HEFNY H A. Artificial intelligence-based traffic flow prediction: a comprehensive review[J]. Journal of Electrical Systems and Information Technology, 2023, 10(1): 13.
[23] AL SAHILI Z, AWAD M. Spatio-temporal graph neural networks: a survey[EB/OL]. [2024-10-23]. https://arxiv.org/abs/ 2301.10569.
[24] JIN M, KOH H Y, WEN Q S, et al. A survey on graph neural networks for time series: forecasting, classification, imputation, and anomaly detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(12): 10466-10485.
[25] WU Z H, PAN S R, LONG G D, et al. Graph WaveNet for deep spatial-temporal graph modeling[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. Palo Alto: AAAI, 2019: 1907-1913.
[26] BAI L, YAO L N, LI C, et al. Adaptive graph convolutional recurrent network for traffic forecasting[C]//Advances in Neural Information Processing Systems 33, 2020: 17804-17815.
[27] WU Z H, PAN S R, LONG G D, et al. Connecting the Dots: multivariate time series forecasting with graph neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2020: 753-763.
[28] LI F X, FENG J, YAN H, et al. Dynamic graph convolutional recurrent network for traffic prediction: benchmark and solution[J]. ACM Transactions on Knowledge Discovery from Data, 2023, 17(1): 1-21.
[29] KONG W Y, GUO Z Y, LIU Y B. Spatio-temporal pivotal graph neural networks for traffic flow forecasting[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(8): 8627-8635.
[30] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30, 2017: 5998-6008.
[31] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understan-ding[EB/OL]. [2024-10-23]. https://arxiv.org/abs/1810.04805.
[32] GUO S N, LIN Y F, WAN H Y, et al. Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(11): 5415-5428.
[33] XU M X, DAI W R, LIU C M, et al. Spatial-temporal transformer networks for traffic flow forecasting[EB/OL]. [2024-10-23]. https://arxiv.org/abs/2001.02908.
[34] SONG C, LIN Y F, GUO S N, et al. Spatial-temporal synchronous graph convolutional networks: a new framework for spatial-temporal network data forecasting[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(1): 914-921.