Deep Exponential Moving Average Learning Method for Sequential Recommendation

doi:10.3778/j.issn.1673-9418.2403068

Abstract

Abstract: Sequential recommendation is a recommendation paradigm that models the user’s historical interaction sequence to predict the user􀆳s preference for the next item. Due to the diversity and complexity of user interaction history, interaction sequences often contain noisy and irrelevant noise terms, which affects the model to capture stable sequence patterns. At the same time, there are various sequence signals in the interactive sequence, such as trend signal, semantic signal and residual signal, which cannot be accurately expressed by a single type of modeling. To solve these two problems, this paper proposes a deep exponential moving average based learning method for sequential recommendation (DeepEMA). In order to solve the noise problem in user interaction sequences, DeepEMA uses the exponential moving average method in the field of time series to smooth the sequences, filter the noise and initially extract the sequence trend. In order to capture complex and diverse sequence signals in interaction sequences, DeepEMA models the sequential trend signal, dimensional semantic signal, and residual signal respectively through a multi-module architecture including trend module, dimensional semantic module, and residual extraction module. Experimental results on four publicly available datasets demonstrate that, the proposed model demonstrates superior recommendation accuracy in terms of hit ratio (HR) and normalized discounted cumulative gain (NDCG) metrics. For example, there is an average improvement of 5.67% on the HR@5 metric and 4.88% on the NDCG@5 metric compared with the most advanced baseline. And the sequence noise problem is significantly improved. The ablation experiments also verify the effectiveness of each module.

Key words: sequential recommendation, moving average, noise, sequential signal

摘要： 序列推荐是对用户历史交互序列进行建模，以预测用户所偏好的下一项目的推荐范式。由于用户交互历史的多样性和复杂性，交互序列常包含嘈杂和不相关的噪声项，从而影响模型捕捉稳定的序列模式；同时，交互序列中存在多样的序列信号，如趋势信号、语义信号和残差信号等，依赖单一模式进行建模无法得到精确的表达。针对上述两个问题，提出了一种基于深度化指数移动平均的序列推荐模型（DeepEMA）。为了解决用户交互序列中的噪音问题，DeepEMA使用时间序列领域的指数移动平均法对序列进行平滑处理，过滤噪音并初步提取序列趋势；为了捕捉交互序列中复杂多样的序列信号，DeepEMA通过包括序列趋势模块、维度语义模块及残差提取模块在内的多模块架构，分别建模序列的趋势信号、维度的语义信号以及残差信号。在四个公开的数据集上进行了验证，实验结果表明，该模型的推荐准确性在命中率（HR）和归一化折损累计增益（NDCG）指标上均优于最先进的基线算法。在HR@5指标上，比最先进的基线平均提高了5.67%；在NDCG@5指标上，比最先进的基线平均提高了4.88%。序列的噪音问题得到了明显改善，消融实验也验证了各模块的有效性。

关键词: 序列推荐, 移动平均, 噪音, 序列信号

WEN Wen, HU Zhixin, HAO Zhifeng. Deep Exponential Moving Average Learning Method for Sequential Recommendation[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(3): 774-786.

温雯, 胡智鑫, 郝志峰. 面向序列推荐的深度化指数移动平均学习方法[J]. 计算机科学与探索, 2025, 19(3): 774-786.

References

[1] FANG H, ZHANG D N, SHU Y H, et al. Deep learning for sequential recommendation: algorithms, influential factors, and evaluations[J]. ACM Transactions on Information Systems, 2020, 39(1): 1-42.
[2] SARWAR B, KARYPIS G, KONSTAN J, et al. Item-based collaborative filtering recommendation algorithms[C]//Proceedings of the 10th International Conference on World Wide Web. New York: ACM, 2001: 285-295.
[3] WANG S J, HU L, WANG Y, et al. Sequential recommender systems: challenges, progress and prospects[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019: 6332-6338.
[4] SAID A, JAIN B J, NARR S, et al. Users and noise: the magic barrier of recommender systems[C]//Proceedings of the 20th International Conference on User Modeling, Adaptation, and Personalization. Berlin, Heidelberg: Springer, 2012: 237-248.
[5] LEVER J, KRZYWINSKI M, ALTMAN N. Model selection and overfitting[J]. Nature Methods, 2016, 13: 703-704.
[6] CEN Y K, ZHANG J W, ZOU X, et al. Controllable multi-interest framework for recommendation[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2020: 2942-2951.
[7] RENDLE S, FREUDENTHALER C, SCHMIDT-THIEME L, et al. Factorizing personalized Markov chains for next-basket recommendation[C]//Proceedings of the 19th International Conference on World Wide Web. New York: ACM, 2010: 811-820.
[8] TANG J X, WANG K, TANG J X, et al. Personalized top-N sequential recommendation via convolutional sequence embedding[C]//Proceedings of the 11th ACM International Conference on Web Search and Data Mining. New York: ACM, 2018: 565-573.
[9] KANG W C, MCAULEY J. Self-attentive sequential recommendation[C]//Proceedings of the 2018 IEEE International Conference on Data Mining. Piscataway: IEEE, 2018: 197-206.
[10] SUN F, LIU J, WU J, et al. BERT4Rec[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: ACM, 2019: 1441-1450.
[11] 余文婷, 吴云. 时间感知的双塔型自注意力序列推荐模型[J]. 计算机科学与探索, 2024, 18(1): 175-188.
YU W T, WU Y. Time-aware sequential recommendation model based on dual-tower self-attention[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(1): 175-188.
[12] SHIN Y, CHOI J, WI H, et al. An attentive inductive bias for sequential recommendation beyond the self-attention[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(8): 8984-8992.
[13] YANG Y H, HUANG C, XIA L H, et al. Debiased contrastive learning for sequential recommendation[C]//Proceedings of the ACM Web Conference 2023. New York: ACM, 2023: 1063-1073.
[14] QIN X Y, YUAN H H, ZHAO P P, et al. Meta-optimized contrastive learning for sequential recommendation[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2023: 89-98.
[15] 吴静, 谢辉, 姜火文. 图神经网络推荐系统综述[J]. 计算机科学与探索, 2022, 16(10): 2249-2263.
WU J, XIE H, JIANG H W. Survey of graph neural network in recommendation system[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(10): 2249-2263.
[16] HUANG L W, MA Y T, LIU Y B, et al. Position-enhanced and time-aware graph convolutional network for sequential recommendations[J]. ACM Transactions on Information Systems, 2023, 41(1): 1-32.
[17] 陈万志, 王军. 时间感知增强的动态图神经网络序列推荐算法[J]. 计算机工程与应用, 2024, 60(20): 142-152.
CHEN W Z, WANG J. Time-aware enhancement dynamic graph neural networks for sequential recommendation algorithm[J]. Computer Engineering and Applications, 2024, 60(20): 142-152.
[18] YE Y W, XIA L H, HUANG C, et al. Graph masked autoencoder for sequential recommendation[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2023: 321-330.
[19] LIU H, DAI Z, SO D, et al. Pay attention to MLPs[C]//Advances in Neural Information Processing Systems 34, Dec 6-14, 2021: 9204-9215.
[20] LI M Y, ZHAO X Y, LYU C, et al. MLP4Rec: a pure MLP architecture for sequential recommendations[C]//Proceedings of the 31st International Joint Conference on Artificial Intelligence, 2022: 2138-2144.
[21] GAO J T, ZHAO X Y, LI M Y, et al. SMLP4Rec: an efficient all-MLP architecture for sequential recommendations[J]. ACM Transactions on Information Systems, 2024, 42(3): 1-23.
[22] LI M Y, ZHANG Z J, ZHAO X Y, et al. AutoMLP: automated MLP for sequential recommendations[C]//Proceedings of the ACM Web Conference 2023. New York: ACM, 2023: 1190-1198.
[23] LIANG J H, ZHAO X Y, LI M Y, et al. MMMLP: multi-modal multilayer perceptron for sequential recommendations[C]//Proceedings of the ACM Web Conference 2023. New York: ACM, 2023: 1109-1117.
[24] QIN Y Q, WANG P F, LI C L, et al. The world is binary: contrastive learning for denoising next basket recommendation[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2021: 859-868.
[25] TONG X H, WANG P F, LI C L, et al. Pattern-enhanced contrastive policy learning network for sequential recommendation[C]//Proceedings of the 30th International Joint Conference on Artificial Intelligence, 2021: 1593-1599.
[26] ZHOU K, YU H, ZHAO W X, et al. Filter-enhanced MLP is all you need for sequential recommendation[C]//Proceedings of the ACM Web Conference 2022. New York: ACM, 2022: 2388-2399.
[27] LIN Y J, WANG C Y, CHEN Z M, et al. A self-correcting sequential recommender[C]//Proceedings of the ACM Web Conference 2023. New York: ACM, 2023: 1283-1293.
[28] ZHOU P L, YE Q C, XIE Y Q, et al. Attention calibration for transformer-based sequential recommendation[C]//Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. New York: ACM, 2023: 3595-3605.
[29] ROTELA P JR, SALOMON F L R, DE OLIVEIRA PAMPLONA E. ARIMA: an applied time series forecasting model for the Bovespa Stock Index[J]. Applied Mathematics, 2014, 5(21): 3383-3391.
[30] SENG H S, CHARLES V, GHERMAN T, et al. Hull-WEMA: a novel zero-lag approach in the moving average family, with an application to COVID-19[J]. International Journal of Management and Decision Making, 2022, 21(1): 92.
[31] WINTERS P R. Forecasting sales by exponentially weighted moving averages[J]. Management Science, 1960, 6(3): 324-342.
[32] HUNTER J S. The exponentially weighted moving average[J]. Journal of Quality Technology, 1986, 18(4): 203-210.
[33] SVETUNKOV I, KOURENTZES N, ORD J K. Complex exponential smoothing[J]. Naval Research Logistics, 2022, 69(8): 1108-1123.
[34] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
[35] HARPER F M, KONSTAN J A. The MovieLens datasets: history and context[J]. ACM Transactions on Interactive Intelligent Systems, 2015, 5(4): 1-19.
[36] MCAULEY J, TARGETT C, SHI Q F, et al. Image-based recommendations on styles and substitutes[C]//Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2015: 43-52.
[37] CHO J, HYUN D, KANG S, et al. Learning heterogeneous temporal patterns of user preference for timely recommendation[C]//Proceedings of the Web Conference 2021. New York: ACM, 2021: 1274-1283.
[38] YUE Z R, WANG Y Q, HE Z K, et al. Linear recurrent units for sequential recommendation[C]//Proceedings of the 17th ACM International Conference on Web Search and Data Mining. New York: ACM, 2024: 930-938.