结合注意力与循环神经网络的专家推荐算法

doi:10.3778/j.issn.1673-9418.2102067

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (9): 2068-2077.DOI: 10.3778/j.issn.1673-9418.2102067

结合注意力与循环神经网络的专家推荐算法

吕晓琦¹^,², 纪科¹^,²^,⁺(), 陈贞翔¹^,², 孙润元¹^,², 马坤¹^,², 邬俊³, 李浥东³

1.济南大学信息科学与工程学院,济南 250022
2.济南大学山东省网络环境智能计算技术重点实验室,济南 250022
3.北京交通大学计算机与信息技术学院,北京 100044

收稿日期:2021-01-20 修回日期:2021-03-18 出版日期:2022-09-01 发布日期:2021-03-26
通讯作者: + E-mail: ise_jik@ujn.edu.cn
作者简介:吕晓琦（1997—）,女,山东滨州人,硕士研究生,主要研究方向为自然语言处理。
纪科（1989—）,男,辽宁辽阳人,博士,副教授,主要研究方向为机器学习、推荐系统等。
陈贞翔（1979—）,男,湖南郴州人,博士,教授,主要研究方向为互联网测量、网络安全。
孙润元（1976—）,男,黑龙江哈尔滨人,硕士,副教授,主要研究方向为大数据、云计算等。
马坤（1981—）,男,山东济南人,博士,副教授,主要研究方向为大数据、云计算等。
邬俊（1981—）,男,辽宁鞍山人,博士,副教授,主要研究方向为机器学习、人工智能等。
李浥东（1982—）, 男, 山西太原人, 博士, 教授, 主要研究方向为大数据分析、人工智能等。
基金资助:
国家自然科学基金(61702216);国家自然科学基金(61772231);国家自然科学基金(61671048);国家自然科学基金(61672262);山东省重大科技创新工程(2018CXGC0706)

Expert Recommendation Algorithm Combining Attention and Recurrent Neural Network

LYU Xiaoqi¹^,², JI Ke¹^,²^,⁺(), CHEN Zhenxiang¹^,², SUN Runyuan¹^,², MA Kun¹^,², WU Jun³, LI Yidong³

1. School of Information Science and Engineering, University of Jinan, Jinan 250022, China
2. Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan 250022, China
3. School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China

Received:2021-01-20 Revised:2021-03-18 Online:2022-09-01 Published:2021-03-26
About author:LYU Xiaoqi, born in 1997, M.S. candidate. Her research interest is natural language processing.
JI Ke, born in 1989, Ph.D., associate professor. His research interests include machine learning, recommendation system, etc.
CHEN Zhenxiang, born in 1979, Ph.D., professor. His research interests include network measurement and network security.
SUN Runyuan, born in 1976, M.S., associate professor. His research interests include big data, cloud computing, etc.
MA Kun, born in 1981, Ph.D., associate professor. His research interests include big data, cloud computing, etc.
WU Jun, born in 1981, Ph.D., associate professor. His research interests include machine learning, artificial intelligence, etc.
LI Yidong, born in 1982, Ph.D., professor. His research interests include big data analysis, artificial intelligence, etc.
Supported by:
National Natural Science Foundation of China(61702216);National Natural Science Foundation of China(61772231);National Natural Science Foundation of China(61671048);National Natural Science Foundation of China(61672262);Key Research and Development Program of Shandong Province(2018CXGC0706)

摘要/Abstract

摘要：

在线问答社区（CQA）已经成为互联网最重要的知识分享交流平台,将用户提出的海量问题有效推荐给可能解答的用户,挖掘用户感兴趣的问题是此类平台最核心功能。一些针对问答社区的专家推荐算法已经被提出用来提高平台解答效率,但是现有工作大多关注于用户兴趣与问题信息匹配,忽视了用户兴趣动态变化问题,可能会严重影响推荐质量。提出了结合注意力与循环神经网络的专家推荐算法,不仅实现了问题信息的深度特征编码,而且还能捕获动态变化的用户兴趣。首先,问题编码器在预训练词嵌入基础上结合卷积神经网络（CNN）和Attention注意力机制实现了问题标题与绑定标签的深度特征联合表示。然后,用户编码器在用户历史回答问题的时间序列上利用长短期记忆神经网络Bi-GRU模型捕捉动态兴趣,并结合用户固定标签信息表征长期兴趣。最后,根据两个编码器输出向量的相似性计算产生用户动态兴趣与长期兴趣相结合的推荐结果。在来自知乎问答社区的真实数据上进行了不同参数配置及不同算法的对比实验,结果表明该算法性能明显优于目前比较流行的深度学习专家推荐算法。

关键词: 社区问答（CQA）, 专家推荐, 深度学习, 注意力机制, 循环神经网络（RNN）

Abstract:

Community question answering (CQA) has become the most important knowledge sharing and exchange platform on the Internet, effectively recommending massive questions raised by users to users who may answer them, and mining the questions that users are interested in is the core function of such platforms. Some expert recommendation algorithms for CQA have been proposed to improve the efficiency of platform answering, but most of the existing work focuses on matching user interests with question information, ignoring the dynamic change of user interests, which may seriously affect the quality of recommendation. This paper proposes an expert recommen-dation algorithm combining attention and recurrent neural network (RNN), which not only realizes deep feature coding of the question content, but also captures the dynamically changing user interest. First, the question encoder combines convolutional neural network (CNN) and Attention mechanism on the basis of pre-trained word embeddings to realize joint representation of deep feature of question title and bound topics. Then, the user encoder captures the dynamic interest using Bi-GRU model on the time series of user's historical answers to questions, and combines the user's fixed label information to represent the long-term interest. Finally, a recommendation result combining the user's dynamic interest and long-term interest is generated according to the similarity calculation of output vectors of two encoders. This paper conducts comparative experiments on different parameter configurations and different algorithms on real data from Zhihu. Experimental results show that the performance of the algorithm is significantly better than the current popular deep learning expert recommendation methods.

Key words: community question answering (CQA), expert recommendation, deep learning, attention mechanism, recurrent neural network (RNN)

中图分类号:

TP391

吕晓琦, 纪科, 陈贞翔, 孙润元, 马坤, 邬俊, 李浥东. 结合注意力与循环神经网络的专家推荐算法[J]. 计算机科学与探索, 2022, 16(9): 2068-2077.

LYU Xiaoqi, JI Ke, CHEN Zhenxiang, SUN Runyuan, MA Kun, WU Jun, LI Yidong. Expert Recommendation Algorithm Combining Attention and Recurrent Neural Network[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2068-2077.

图/表 10

图1 问题编码器结构

Fig.1 Structure of question encoder

图2 用户编码器结构

Fig.2 Structure of user encoder

图3 GRU模型结构

Fig.3 Structure of GRU model

图4 判断当前用户是否会接受此问题

Fig.4 Determine whether current user will accept the given question

表1 数据集基本统计

Table 1 Basic statistics of dataset

名称	数量
用户描述记录	1 931 654
问题描述记录	1 829 900
问题邀请记录	500 000
主题标签	100 000

图5 不同全连接层结构的性能变化

Fig.5 Performance changes under different FC layers

图6 不同Epoch的性能变化

Fig.6 Performance changes under different epochs

图7 不同比例训练数据的性能变化

Fig.7 Performance changes under different proportions of training data

图8 不同输入信息对问题编码器的影响

Fig.8 Influence of different input information on question encoder

表2 不同比例训练数据上所有算法的实验结果比较

Table 2 Comparison of experimental results of all algorithms on different proportions of training data

算法名称	AUC			ACC			Logloss
算法名称	30%	70%	100%	30%	70%	100%	30%	70%	100%
DeepFM	0.522 3	0.530 1	0.532 3	0.724 9	0.731 8	0.741 0	0.466 6	0.466 8	0.468 3
XDeepFM	0.574 1	0.576 0	0.584 7	0.754 0	0.766 0	0.788 6	0.462 9	0.461 8	0.460 7
CNN-DSSM	0.584 8	0.605 9	0.613 3	0.787 5	0.800 5	0.801 8	0.457 5	0.449 2	0.449 0
DSIERM-OS	0.628 8	0.638 5	0.645 6	0.801 2	0.809 0	0.811 2	0.451 0	0.439 1	0.437 1
DSIERM	0.630 2	0.647 6	0.649 1	0.809 6	0.810 2	0.811 6	0.439 1	0.431 7	0.431 0

参考文献 25

[1]	WANG X, HUANG C, YAO L, et al. A survey on expert recommendation in community question answering[J]. Journal of Computer Science and Technology, 2018, 33(4): 625-653. DOI URL
[2]	NIKZAD-KHASMAKHI N, BALAFAR M A, FEIZI-DERA-KHSHI M R. The state-of-the-art in expert recommendation systems[J]. Engineering Applications of Artificial Intelligence, 2019, 82: 126-147. DOI URL
[3]	LIU X Y, CROFT W B, KOLL M B. Finding experts in community-based question-answering services[C]// Procee-dings of the 2005 ACM CIKM International Conference on Information and Knowledge Management, Bremen, Oct 31-Nov 5, 2005. New York: ACM, 2005: 315-316.
[4]	LIU M R, LIU Y C, YANG Q. Predicting best answerers for new questions in community question answering[C]// LNCS 6184: Proceedings of the 11th International Conference on Web-Age Information Management, Jiuzhaigou, Jul 15-17, 2010. Berlin, Heidelberg: Springer, 2010: 127-138.
[5]	SHEN Y K, RONG W G, SUN Z W, et al. Question/Answer matching for CQA system via combining lexical and seque-ntial information[C]// Proceedings of the 29th AAAI Confe-rence on Artificial Intelligence, Austin, Jan 25-30, 2015. Menlo Park: AAAI, 2015: 275-281.
[6]	YANG L, QIU M H, GOTTIPATI S, et al. CQArank: jointly model topics and expertise in community question answe-ring[C]// Proceedings of the 22nd ACM International Confe-rence on Information and Knowledge Management, San Fran-cisco, Oct 27-Nov 1, 2013. New York: ACM, 2013: 99-108.
[7]	GUO H, TANG R, YE Y, et al. DeepFM: a factorization-machine based neural network for CTR prediction[C]// Procee-dings of the 26th International Joint Conference on Artifi-cial Intelligence. New York: ACM, 2017: 1725-1731.
[8]	LIAN J, ZHOU X, ZHANG F, et al. XDeepFM: combining explicit and implicit feature interactions for recommender systems[C]// Proceedings of the 24th ACM SIGKDD Interna-tional Conference on Knowledge Discovery and Data Mining. New York: ACM, 2018: 1754-1763.
[9]	SHEN Y, HE X, GAO J, et al. A latent semantic model with convolutional-pooling structure for information retrieval[C]// Proceedings of the 23rd ACM International Conference on Information and Knowledge Management. New York: ACM, 2014: 101-110.
[10]	YIN H, ZHOU X, CUI B, et al. Adapting to user interest drift for POI recommendation[J]. IEEE Transactions on Know-ledge and Data Engineering, 2016, 28(10): 2566-2581.
[11]	ZHANG S, YAO L, SUN A, et al. Deep learning based reco-mmender system: a survey and new perspectives[J]. ACM Computing Surveys, 2019, 52(1): 1-38.
[12]	DAS D, SAHOO L, DATTA S. A survey on recommen-dation system[J]. International Journal of Computer Appli-cations, 2017, 160(7): 6-10.
[13]	WU H, WANG Y J, CHENG X. Incremental probabilistic latent semantic analysis for automatic question recommen-dation[C]// Proceedings of the 2008 ACM Conference on Recommender Systems, Lausanne, Oct 23-25, 2008. New York: ACM, 2008: 99-106.
[14]	PAL A, KONSTAN J A. Expert identification in community question answering: exploring question selection bias[C]// Proceedings of the 19th ACM Conference on Information and Knowledge Management, Toronto, Oct 26-30, 2010. New York: ACM, 2010: 1505-1508.
[15]	HUANG P S, HE X D, GAO J F, et al. Learning deep struc-tured semantic models for web search using clickthrough data[C]// Proceedings of the 22nd ACM International Confe-rence on Information and Knowledge Management, San Fran-cisco, Oct 27-Nov 1, 2013. New York: ACM, 2013: 2333-2338.
[16]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Confe-rence on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[17]	KE H, CHEN D, LI X, et al. Towards brain big data classifi-cation: epileptic EEG identification with a lightweight VGGNet on global MIC[J]. IEEE Access, 2018, 6: 14722-14733. DOI URL
[18]	GUO B, ZHANG C, LIU J, et al. Improving text classifi-cation with weighted word embeddings via a multi-channel TextCNN model[J]. Neurocomputing, 2019, 363: 366-374. DOI URL
[19]	KARIMI E, MAJIDI B, MANZURI M T. Relevant ques-tion answering in community based networks using deep LSTM neural networks[C]// Proceedings of the 2019 7th Ira-nian Joint Congress on Fuzzy and Intelligent Systems. Pisca-taway: IEEE, 2019: 1-5.
[20]	YANG M, TU W, QU Q, et al. Advanced community question answering by leveraging external knowledge and multitask learning[J]. Knowledge-Based Systems, 2019, 171: 106-119. DOI URL
[21]	DENG Y, LAM W, XIE Y, et al. Joint learning of answer selection and answer summary generation in community ques-tion answering[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2020: 7651-7658.
[22]	LIU J, YANG Y, LV S, et al. Attention-based BiGRU-CNN for Chinese question classification[J]. Journal of Ambient Inte-lligence and Humanized Computing, 2019. DOI: 10.1007/s12652-019-01344-9. DOI
[23]	ZHANG Y, LU W, OU W, et al. Chinese medical question ans-wer selection via hybrid models based on CNN and GRU[J]. Multimedia Tools and Applications, 2020, 79(21): 14751-14776. DOI URL
[24]	YUAN S, ZHANG Y, TANG J, et al. Expert finding in commu-nity question answering: a review[J]. Artificial Intelligence Review, 2020, 53(2): 843-874. DOI URL
[25]	FU J, LI Y, ZHANG Q, et al. Recurrent memory reasoning network for expert finding in community question answering[C]// Proceedings of the 13th International Conference on Web Search and Data Mining. New York: ACM, 2020: 187-195.

结合注意力与循环神经网络的专家推荐算法

Expert Recommendation Algorithm Combining Attention and Recurrent Neural Network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 25

相关文章 15

编辑推荐

Metrics

[1]	李珍琦, 王晶, 贾子钰, 林友芳. 融合注意力的多维特征图卷积运动想象分类[J]. 计算机科学与探索, 2022, 16(9): 2050-2060.
[2]	张祥平, 刘建勋. 基于深度学习的代码表征及其应用综述[J]. 计算机科学与探索, 2022, 16(9): 2011-2029.
[3]	李冬梅, 罗斯斯, 张小平, 许福. 命名实体识别方法研究综述[J]. 计算机科学与探索, 2022, 16(9): 1954-1968.
[4]	任宁, 付岩, 吴艳霞, 梁鹏举, 韩希. 深度学习应用于目标检测中失衡问题研究综述[J]. 计算机科学与探索, 2022, 16(9): 1933-1953.
[5]	杨才东, 李承阳, 李忠博, 谢永强, 孙方伟, 齐锦. 深度学习的图像超分辨率重建技术综述[J]. 计算机科学与探索, 2022, 16(9): 1990-2010.
[6]	曾凡智, 许露倩, 周燕, 周月霞, 廖俊玮. 面向智慧教育的知识追踪模型研究综述[J]. 计算机科学与探索, 2022, 16(8): 1742-1763.
[7]	杨知桥, 张莹, 王新杰, 张东波, 王玉. 改进U型网络在视网膜病变检测中的应用研究[J]. 计算机科学与探索, 2022, 16(8): 1877-1884.
[8]	安凤平, 李晓薇, 曹翔. 权重初始化-滑动窗口CNN的医学图像分类[J]. 计算机科学与探索, 2022, 16(8): 1885-1897.
[9]	夏鸿斌, 肖奕飞, 刘渊. 融合自注意力机制的长文本生成对抗网络模型[J]. 计算机科学与探索, 2022, 16(7): 1603-1610.
[10]	刘艺, 李蒙蒙, 郑奇斌, 秦伟, 任小广. 视频目标跟踪算法综述[J]. 计算机科学与探索, 2022, 16(7): 1504-1515.
[11]	赵小明, 杨轶娇, 张石清. 面向深度学习的多模态情感识别研究进展[J]. 计算机科学与探索, 2022, 16(7): 1479-1503.
[12]	孙方伟, 李承阳, 谢永强, 李忠博, 杨才东, 齐锦. 深度学习应用于遮挡目标检测算法综述[J]. 计算机科学与探索, 2022, 16(6): 1243-1259.
[13]	刘雅芬, 郑艺峰, 江铃燚, 李国和, 张文杰. 深度半监督学习中伪标签方法综述[J]. 计算机科学与探索, 2022, 16(6): 1279-1290.
[14]	李运寰, 闻继伟, 彭力. 高帧率的轻量级孪生网络目标跟踪[J]. 计算机科学与探索, 2022, 16(6): 1405-1416.
[15]	张雁操, 赵宇海, 史岚. 融合图注意力的多特征链接预测算法[J]. 计算机科学与探索, 2022, 16(5): 1096-1106.