Multi-task Recommendation Model of Dual Perception Gated Interaction

doi:10.3778/j.issn.1673-9418.2109017

Abstract

Abstract: Aiming at the problem of negative migration in multi-task recommendation, the multi-task recommen-dation model of dual perception gated interaction (DPGI-MTRM) is proposed. Firstly, in the multi-task sharing network and the proprietary network, the dual-sensing feature extraction module (called the dual-sensing expert layer) is innovatively designed. Its function is to obtain the element-level and vector-level dual-sensing feature representation for the input features. Secondly, a task interaction layer is proposed on the basis of the gated network, which interactively calculates the features output by the gated network to extract high-level semantic relevance between tasks, and at the same time uses the residual method plus the original input gated feature vector to reduce possible noise interference caused by task interaction. Finally, by stacking a dual perception expert layer and a gated interaction layer, and then connecting the neural network output layer of a specific task, a multi-task recommendation model of dual perception gated interaction is obtained. In addition, the multi-objective optimization method of gradient normalization is used during model training, so that the model can better converge. Experiments are conducted on the Census-income, Synthetic Data and Ali-CCP datasets, and the AUC (area under curve) and MSE (mean square error) indicators are used for evaluation. Experimental results show that the proposed model performs better than other benchmark models and achieves more advanced performance.

Key words: multi-task, dual perception expert layer, gated interaction layer, recommendation model

摘要： 针对多任务推荐中存在负迁移的问题，提出一种双感知门控交互的多任务推荐模型（DPGI-MTRM）。首先，在多任务共享网络和专有网络中，创新性地设计了双感知特征提取模块（称为双感知专家层），其作用是对输入特征得到元素级和向量级的双感知特征表示。其次，在门控网络的基础上提出了任务交互层，对经过门控网络输出的特征进行交互计算提取任务之间的高级语义相关性，同时采用残差方式加上原始输入门控的特征向量减少任务交互可能带来的噪音干扰。最后，通过堆叠双感知专家层、门控交互层，连接特定任务的神经网络输出层得到双感知门控交互的多任务推荐模型。此外，在模型训练时使用了梯度归一化的多目标优化方法，使该模型更好地收敛。在Census-income、Synthetic Data和Ali-CCP数据集上进行实验，采用AUC和MSE指标进行评估，实验结果表明，提出的模型表现优于其他基准模型，达到较为先进的性能。

关键词: 多任务, 双感知专家层, 门控交互层, 推荐模型

LIN Jian, WU Yun, CHEN Yukang. Multi-task Recommendation Model of Dual Perception Gated Interaction[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(6): 1417-1426.

林建, 吴云, 陈育康. 双感知门控交互的多任务推荐模型[J]. 计算机科学与探索, 2023, 17(6): 1417-1426.

References

[1] 耿蕾蕾，崔超然，石成，等. 基于深度多任务学习的社交图像标签和分组联合推荐[J]. 计算机科学, 2020, 47(12): 177-182.
GENG L L, CUI C R, SHI C, et al. Social image tag and group joint recommendation based on deep multi-task lear-ning[J]. Computer Science, 2020, 47(12): 177-182.
[2] MA J Q, ZHAO Z, YI X Y, et al. Modeling task relation-ships in multi-task learning with multi-gate mixture-of-experts[C]//Proceedings of the 24th ACM SIGKDD Inter-national Conference on Knowledge Discovery and Data Mining, London, Aug 19-23, 2018. New York: ACM, 2018: 1930-1939.
[3] CARUANA R. Multitask learning[J]. Machine Learning, 1997, 28(1): 41-75.
[4] TANG H Y, LIU J N, ZHAO M, et al. Progressive layered extraction (PLE): a novel multi-task learning (MTL) model for personalized recommendations[C]//Proceedings of the 14th ACM Conference on Recommender Systems, Brazil, Sep 22-26, 2020. New York: ACM, 2020: 269-278.
[5] BANSAL T, BELANGER D, MCCALLUM A. Ask the GRU: multi-task learning for deep text recommendations[C]//Pro-ceedings of the 10th ACM Conference on Recommender Systems, Boston, Sep 15-19, 2016. New York: ACM, 2016: 107-114.
[6] CHENG H T, KOC L, HARMSEN J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 10th ACM Conference on Recommender Systems, Boston, Sep 15-19, 2016. New York: ACM, 2016: 7-10.
[7] ZHOU G R, ZHU X Q, SONG C R, et al. Deep interest net-work for click-through rate prediction[C]//Proceedings of the 24th ACM SIGKDD International Conference on Know-ledge Discovery and Data Mining, Aug 19-23, 2018. New York: ACM, 2018: 1059-1068.
[8] COVINGTON P, ADAMS J, SARGIN E. Deep neural net-works for YouTube recommendations[C]//Proceedings of the 10th ACM Conference on Recommender Systems, Bos-ton, Sep 15-19, 2016. New York: ACM, 2016: 191-198.
[9] ZHU H, JIN J, TAN C, et al. Optimized cost per click in Taobao display advertising[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, Aug 13-17, 2017. New York: ACM, 2017: 2191-2200.
[10] GUO H, TANG R, YE Y, et al. DeepFM: a factorization-machine based neural network for CTR prediction[J]. arXiv:1703.04247, 2017.
[11] LU Y C, DONG R H, SMYTH B. Why I like it: multi-task learning for recommendation and explanation[C]//Procee-dings of the 12th ACM Conference on Recommender Sys-tems, Vancouver, Aug 20-23, 2018. New York: ACM, 2018: 4-12.
[12] MA X, ZHAO L Q, HUANG G, et al. Entire space multi-task model: an effective approach for estimating post-click conversion rate[C]//Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, Jul 8-12, 2018. New York: ACM, 2018: 1137-1140.
[13] ZHAO Z, HONG L C, WEI L, et al. Recommending what video to watch next: a multitask ranking system[C]//Pro-ceedings of the 13th ACM Conference on Recommender Systems, Copenhagen, Sep 16-20, 2019. New York: ACM, 2019: 43-51.
[14] RUDER S, BINGEL J, AUGENSTEIN I, et al. Sluice net-works: learning what to share between loosely related tasks[J]. arXiv:1705.08142, 2017.
[15] JACOBS R, JORDAN M, NOWLAN S, et al. Adaptive mixtures of local experts[J]. Neural Computation, 2014, 3(1): 79-87.
[16] KENDALL A, GAL Y, CIPOLLA R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Washington: IEEE Computer Society, 2018: 7482-7491.
[17] MILOJKOVIC N, ANTOGNINI D, BERGAMIN G, et al. Multi-gradient descent for multi-objective recommender systems[J]. arXiv:2001.00846, 2019.
[18] YU Y T, WANG Z, YUAN B. An input-aware factorization machine for sparse prediction[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 1466-1472.
[19] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Proces-sing Systems 30, Long Beach, Dec 3-9, 2017. Red Hook: Curran Associates, 2017: 5998-6008.
[20] LIN X, CHEN H J, PEI C H, et al. A pareto-efficient algo-rithm for multiple objective optimization in e-commerce recommendation[C]//Proceedings of the 13th ACM Confe-rence on Recommender Systems, Copenhagen, Aug 21-23, 2019. New York: ACM, 2019: 20-28.
[21] DéSIDéRI J A. Multiple-gradient descent algorithm (MGDA) for multiobjective optimization[J]. Comptes Rendus Mathe-matique, 2012, 350(5/6): 313-318.
[22] MA J Q, ZHAO Z, CHEN J L, et al. SNR: sub-network routing for flexible parameter sharing in multi-task learning[C]//Proceedings of the 33rd AAAI Conference on Arti-ficial Intelligence, the 31st Innovative Applications of Arti-ficial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, Hono-lulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 216-223.