Fusion of Masked Autoencoder for Adaptive Augmentation Sequential Recommendation

doi:10.3778/j.issn.1673-9418.2309042

Abstract

Abstract: In order to address the issue of poor-quality contrast views generated by contrastive learning methods in sequential recommendation tasks, a model called GATSR, which is based on graph attention networks for sequential recommendation, is proposed. Firstly, a global item-item transition graph is created based on all user interaction sequences, combining sequential patterns with global collaborative patterns to provide global context for the item representation. Then, an adaptive graph augmentation module is designed to extract important self-supervised signals based on an adaptive sampling strategy, learning more accurate item representations and effectively avoiding the interference of noise signals. Subsequently, the masked autoencoder module employs re-masking technology to mask to highly semantically related masked items again, enabling the encoder to learn higher-level item representations and achieving the reasonable reconstruction of masked items. Finally, the sequential recommender module integrates position information, global context, and the personalized user interaction sequence to obtain the final item representation and predict the user's future possible interaction items based on the representation, thereby providing more reliable recommendation results for users. Experimental results on the Books, Toys, and Retailrocket datasets show that the recommendation accuracy of the proposed model is superior to the most advanced baseline algorithms in terms of hit ratio (HR) and normalized discounted cumulative gain (NDCG) metrics. For example, it improves by 4.59% on the HR@5 metric and 8.89% on the NDCG@5 metric compared with the most advanced baseline.

Key words: sequential recommendation, adaptive data augmentation, autoencoder, self-supervised learning

摘要： 为了解决序列推荐任务中基于对比学习方法生成的对比视图质量不佳的问题，提出融合掩码自编码器的自适应增强序列推荐模型。基于所有用户交互序列创建全局的项目-项目转换图，将序列模式与全局协作模式相结合，为项目表示提供更多的全局上下文；设计自适应图增强模块，基于自适应采样策略提取重要的自监督信号，学习更准确的项目表示，有效避免噪声信号的干扰；基于掩码的自编码器模块利用重掩码技术对高语义相关性的掩码项目进行二次掩码，使编码器学习更高级的项目表示，实现对掩码项目的合理重构；序列推荐器模块将位置信息、全局上下文与用户个性化交互序列相融合得到最终的项目表示，并根据该表示预测用户未来可能的交互项目，从而为用户提供更可靠推荐结果。在Books、Toys以及Retailrocket数据集上的实验结果表明，该模型的推荐准确性在命中率（HR）和归一化折损累计增益（NDCG）指标上均优于最先进的基线算法，例如在HR@5指标上，比最先进的基线提升4.59%，在NDCG@5指标上，比最先进的基线提升8.89%。

关键词: 序列推荐, 自适应数据增强, 自编码器, 自监督学习

SUN Xiujuan, SUN Fuzhen, LI Pengcheng, WANG Aofei, WANG Shaoqing. Fusion of Masked Autoencoder for Adaptive Augmentation Sequential Recommendation[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(12): 3324-3334.

孙秀娟, 孙福振, 李鹏程, 王澳飞, 王绍卿. 融合掩码自编码器的自适应增强序列推荐[J]. 计算机科学与探索, 2024, 18(12): 3324-3334.

References

[1] HIDASI B, KARATZOGLOU A, BALTRUNAS L, et al. Session-based recommendations with recurrent neural networks[EB/OL]. [2023-07-24]. https://arxiv.org/abs/1511.06939.
[2] KANG W C, MCAULEY J. Self-attentive sequential recommendation[C]//Proceedings of the 2018 IEEE International Conference on Data Mining. Piscataway: IEEE, 2018: 197-206.
[3] CHEN Z, ZHANG W, YAN J, et al. Learning dual dynamic representations on time-sliced user-item interaction graphs for sequential recommendation[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management. New York: ACM, 2021: 231-240.
[4] 韩滕跃, 牛少彰, 张文. 基于对比学习的多模态序列推荐算法[J]. 计算机应用, 2022, 42(6): 1683-1688.
HAN T Y, NIU S Z, ZHANG W. Multimodal sequential recommendation algorithm based on contrastive learning[J]. Journal of Computer Applications, 2022, 42(6): 1683-1688.
[5] QIU R, HUANG Z, YIN H, et al. Contrastive learning for representation degeneration problem in sequential recommendation[C]//Proceedings of the 15th ACM International Conference on Web Search and Data Mining. New York:ACM, 2022: 813-823.
[6] 方阳, 谭真, 陈子阳, 等. 用于冷启动推荐的异质信息网络对比元学习[J]. 软件学报, 2023, 34(10): 4548-4564.
FANG Y, TAN Z, CHEN Z Y, et al. Contrastive meta-learning on heterogeneous information networks for cold-start recommendation[J]. Journal of Software, 2023, 34(10): 4548-4564.
[7] LI J, REN P, CHEN Z, et al. Neural attentive session-based recommendation[C]//Proceedings of the 2017 ACM Conference Information and Knowledge Management. New York: ACM, 2017: 1419-1428.
[8] 陈旭松. 基于用户行为序列建模的推荐算法研究[D]. 合肥: 中国科学技术大学, 2021.
CHEN X S. Recommendation algorithms based on user behavior sequence modeling[D]. Hefei: University of Science and Technology of China, 2021.
[9] 赵港, 王千阁, 姚烽, 等. 大规模图神经网络系统综述[J]. 软件学报, 2022, 33(1): 150-170.
ZHAO G, WANG Q G, YAO F, et al. Survey on large-scale graph neural network systems[J]. Journal of Software, 2022, 33(1): 150-170.
[10] 刘杰, 尚学群, 宋凌云, 等. 图神经网络在复杂图挖掘上的研究进展[J]. 软件学报, 2022, 33(10): 3582-3618.
LIU J, SHANG X Q, SONG L Y, et al. Progress of graph neural networks on complex graph mining[J]. Journal of Software, 2022, 33(10): 3582-3618.
[11] 马帅, 刘建伟, 左信. 图神经网络综述[J]. 计算机研究与发展, 2022, 59(1): 47-80.
MA S, LIU J W, ZUO X. Survey on graph neural network[J]. Journal of Computer Research and Development, 2022, 59(1): 47-80.
[12] WU S, TANG Y, ZHU Y, et al. Session-based recommendation with graph neural networks[C]//Proceedings of the 2019 AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2019: 346-353.
[13] WANG Z, WEI W, CONG G, et al. Global context enhanced graph neural networks for session-based recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 169-178.
[14] CHANG J, GAO C, ZHENG Y, et al. Sequential recommendation with graph neural networks[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM,2021: 378-387.
[15] XIE X, SUN F, LIU Z, et al. Contrastive learning for sequential recommendation[C]//Proceedings of the 2022 IEEE 38th International Conference on Data Engineering. Piscataway: IEEE, 2022: 1259-1273.
[16] LIU Z, CHEN Y, LI J, et al. Contrastive self-supervised sequential recommendation with robust augmentation[EB/OL]. [2023-07-24]. https://arxiv.org/abs/2108.06479.
[17] CHEN Y, LIU Z, LI J, et al. Intent contrastive learning for sequential recommendation[C]//Proceedings of the ACM Web Conference 2022. New York: ACM, 2022: 2172-2182.
[18] 余文婷, 吴云, 林建. 融合多模态自监督图学习的视频推荐模型[J]. 计算机应用研究, 2023, 40(6): 1679-1685.
YU W T, WU Y, LIN J. Self-supervised graph learning of fusing multi-modal for video recommendation model[J]. Application Research of Computers, 2023, 40(6): 1679-1685.
[19] SEDHAIN S, MENON A K, SANNER S, et al. Autorec: autoencoders meet collaborative filtering[C]//Proceedings of the 24th International Conference on World Wide Web. New York: ACM, 2015: 111-112.
[20] SUN F, LIU J, WU J, et al. BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: ACM, 2019: 1441-1450.
[21] WANG J, DING K, HONG L, et al. Next-item recommendation with sequential hypergraphs[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 1101-1110.
[22] WANG C, MA W, CHEN C, et al. Sequential recommendation with multiple contrast signals[J]. ACM Transactions on Information Systems, 2023, 41(1): 1-27.
[23] YE Y, XIA L, HUANG C. Graph masked autoencoder for sequential recommendation[EB/OL]. [2023-07-24]. https://arxiv.org/abs/2305.04619.
[24] YANG Y, HUANG C, XIA L, et al. Debiased contrastive learning for sequential recommendation[C]//Proceedings of the ACM Web Conference 2023. New York: ACM, 2023: 1063-1073.