Expert Recommendation Algorithm Combining Attention and Recurrent Neural Network

doi:10.3778/j.issn.1673-9418.2102067

Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (9): 2068-2077.DOI: 10.3778/j.issn.1673-9418.2102067

• Artificial Intelligence • Previous Articles Next Articles

Expert Recommendation Algorithm Combining Attention and Recurrent Neural Network

LYU Xiaoqi¹^,², JI Ke¹^,²^,⁺(), CHEN Zhenxiang¹^,², SUN Runyuan¹^,², MA Kun¹^,², WU Jun³, LI Yidong³

1. School of Information Science and Engineering, University of Jinan, Jinan 250022, China
2. Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan 250022, China
3. School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China

Received:2021-01-20 Revised:2021-03-18 Online:2022-09-01 Published:2021-03-26
About author:LYU Xiaoqi, born in 1997, M.S. candidate. Her research interest is natural language processing.
JI Ke, born in 1989, Ph.D., associate professor. His research interests include machine learning, recommendation system, etc.
CHEN Zhenxiang, born in 1979, Ph.D., professor. His research interests include network measurement and network security.
SUN Runyuan, born in 1976, M.S., associate professor. His research interests include big data, cloud computing, etc.
MA Kun, born in 1981, Ph.D., associate professor. His research interests include big data, cloud computing, etc.
WU Jun, born in 1981, Ph.D., associate professor. His research interests include machine learning, artificial intelligence, etc.
LI Yidong, born in 1982, Ph.D., professor. His research interests include big data analysis, artificial intelligence, etc.
Supported by:
National Natural Science Foundation of China(61702216);National Natural Science Foundation of China(61772231);National Natural Science Foundation of China(61671048);National Natural Science Foundation of China(61672262);Key Research and Development Program of Shandong Province(2018CXGC0706)

结合注意力与循环神经网络的专家推荐算法

吕晓琦¹^,², 纪科¹^,²^,⁺(), 陈贞翔¹^,², 孙润元¹^,², 马坤¹^,², 邬俊³, 李浥东³

1.济南大学信息科学与工程学院,济南 250022
2.济南大学山东省网络环境智能计算技术重点实验室,济南 250022
3.北京交通大学计算机与信息技术学院,北京 100044

通讯作者: + E-mail: ise_jik@ujn.edu.cn
作者简介:吕晓琦（1997—）,女,山东滨州人,硕士研究生,主要研究方向为自然语言处理。
纪科（1989—）,男,辽宁辽阳人,博士,副教授,主要研究方向为机器学习、推荐系统等。
陈贞翔（1979—）,男,湖南郴州人,博士,教授,主要研究方向为互联网测量、网络安全。
孙润元（1976—）,男,黑龙江哈尔滨人,硕士,副教授,主要研究方向为大数据、云计算等。
马坤（1981—）,男,山东济南人,博士,副教授,主要研究方向为大数据、云计算等。
邬俊（1981—）,男,辽宁鞍山人,博士,副教授,主要研究方向为机器学习、人工智能等。
李浥东（1982—）, 男, 山西太原人, 博士, 教授, 主要研究方向为大数据分析、人工智能等。
基金资助:
国家自然科学基金(61702216);国家自然科学基金(61772231);国家自然科学基金(61671048);国家自然科学基金(61672262);山东省重大科技创新工程(2018CXGC0706)

Abstract

Abstract:

Community question answering (CQA) has become the most important knowledge sharing and exchange platform on the Internet, effectively recommending massive questions raised by users to users who may answer them, and mining the questions that users are interested in is the core function of such platforms. Some expert recommendation algorithms for CQA have been proposed to improve the efficiency of platform answering, but most of the existing work focuses on matching user interests with question information, ignoring the dynamic change of user interests, which may seriously affect the quality of recommendation. This paper proposes an expert recommen-dation algorithm combining attention and recurrent neural network (RNN), which not only realizes deep feature coding of the question content, but also captures the dynamically changing user interest. First, the question encoder combines convolutional neural network (CNN) and Attention mechanism on the basis of pre-trained word embeddings to realize joint representation of deep feature of question title and bound topics. Then, the user encoder captures the dynamic interest using Bi-GRU model on the time series of user's historical answers to questions, and combines the user's fixed label information to represent the long-term interest. Finally, a recommendation result combining the user's dynamic interest and long-term interest is generated according to the similarity calculation of output vectors of two encoders. This paper conducts comparative experiments on different parameter configurations and different algorithms on real data from Zhihu. Experimental results show that the performance of the algorithm is significantly better than the current popular deep learning expert recommendation methods.

Key words: community question answering (CQA), expert recommendation, deep learning, attention mechanism, recurrent neural network (RNN)

摘要：

在线问答社区（CQA）已经成为互联网最重要的知识分享交流平台,将用户提出的海量问题有效推荐给可能解答的用户,挖掘用户感兴趣的问题是此类平台最核心功能。一些针对问答社区的专家推荐算法已经被提出用来提高平台解答效率,但是现有工作大多关注于用户兴趣与问题信息匹配,忽视了用户兴趣动态变化问题,可能会严重影响推荐质量。提出了结合注意力与循环神经网络的专家推荐算法,不仅实现了问题信息的深度特征编码,而且还能捕获动态变化的用户兴趣。首先,问题编码器在预训练词嵌入基础上结合卷积神经网络（CNN）和Attention注意力机制实现了问题标题与绑定标签的深度特征联合表示。然后,用户编码器在用户历史回答问题的时间序列上利用长短期记忆神经网络Bi-GRU模型捕捉动态兴趣,并结合用户固定标签信息表征长期兴趣。最后,根据两个编码器输出向量的相似性计算产生用户动态兴趣与长期兴趣相结合的推荐结果。在来自知乎问答社区的真实数据上进行了不同参数配置及不同算法的对比实验,结果表明该算法性能明显优于目前比较流行的深度学习专家推荐算法。

关键词: 社区问答（CQA）, 专家推荐, 深度学习, 注意力机制, 循环神经网络（RNN）

CLC Number:

TP391

LYU Xiaoqi, JI Ke, CHEN Zhenxiang, SUN Runyuan, MA Kun, WU Jun, LI Yidong. Expert Recommendation Algorithm Combining Attention and Recurrent Neural Network[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2068-2077.

吕晓琦, 纪科, 陈贞翔, 孙润元, 马坤, 邬俊, 李浥东. 结合注意力与循环神经网络的专家推荐算法[J]. 计算机科学与探索, 2022, 16(9): 2068-2077.

Figures/Tables 10

References 25

[1]	WANG X, HUANG C, YAO L, et al. A survey on expert recommendation in community question answering[J]. Journal of Computer Science and Technology, 2018, 33(4): 625-653. DOI URL
[2]	NIKZAD-KHASMAKHI N, BALAFAR M A, FEIZI-DERA-KHSHI M R. The state-of-the-art in expert recommendation systems[J]. Engineering Applications of Artificial Intelligence, 2019, 82: 126-147. DOI URL
[3]	LIU X Y, CROFT W B, KOLL M B. Finding experts in community-based question-answering services[C]// Procee-dings of the 2005 ACM CIKM International Conference on Information and Knowledge Management, Bremen, Oct 31-Nov 5, 2005. New York: ACM, 2005: 315-316.
[4]	LIU M R, LIU Y C, YANG Q. Predicting best answerers for new questions in community question answering[C]// LNCS 6184: Proceedings of the 11th International Conference on Web-Age Information Management, Jiuzhaigou, Jul 15-17, 2010. Berlin, Heidelberg: Springer, 2010: 127-138.
[5]	SHEN Y K, RONG W G, SUN Z W, et al. Question/Answer matching for CQA system via combining lexical and seque-ntial information[C]// Proceedings of the 29th AAAI Confe-rence on Artificial Intelligence, Austin, Jan 25-30, 2015. Menlo Park: AAAI, 2015: 275-281.
[6]	YANG L, QIU M H, GOTTIPATI S, et al. CQArank: jointly model topics and expertise in community question answe-ring[C]// Proceedings of the 22nd ACM International Confe-rence on Information and Knowledge Management, San Fran-cisco, Oct 27-Nov 1, 2013. New York: ACM, 2013: 99-108.
[7]	GUO H, TANG R, YE Y, et al. DeepFM: a factorization-machine based neural network for CTR prediction[C]// Procee-dings of the 26th International Joint Conference on Artifi-cial Intelligence. New York: ACM, 2017: 1725-1731.
[8]	LIAN J, ZHOU X, ZHANG F, et al. XDeepFM: combining explicit and implicit feature interactions for recommender systems[C]// Proceedings of the 24th ACM SIGKDD Interna-tional Conference on Knowledge Discovery and Data Mining. New York: ACM, 2018: 1754-1763.
[9]	SHEN Y, HE X, GAO J, et al. A latent semantic model with convolutional-pooling structure for information retrieval[C]// Proceedings of the 23rd ACM International Conference on Information and Knowledge Management. New York: ACM, 2014: 101-110.
[10]	YIN H, ZHOU X, CUI B, et al. Adapting to user interest drift for POI recommendation[J]. IEEE Transactions on Know-ledge and Data Engineering, 2016, 28(10): 2566-2581.
[11]	ZHANG S, YAO L, SUN A, et al. Deep learning based reco-mmender system: a survey and new perspectives[J]. ACM Computing Surveys, 2019, 52(1): 1-38.
[12]	DAS D, SAHOO L, DATTA S. A survey on recommen-dation system[J]. International Journal of Computer Appli-cations, 2017, 160(7): 6-10.
[13]	WU H, WANG Y J, CHENG X. Incremental probabilistic latent semantic analysis for automatic question recommen-dation[C]// Proceedings of the 2008 ACM Conference on Recommender Systems, Lausanne, Oct 23-25, 2008. New York: ACM, 2008: 99-106.
[14]	PAL A, KONSTAN J A. Expert identification in community question answering: exploring question selection bias[C]// Proceedings of the 19th ACM Conference on Information and Knowledge Management, Toronto, Oct 26-30, 2010. New York: ACM, 2010: 1505-1508.
[15]	HUANG P S, HE X D, GAO J F, et al. Learning deep struc-tured semantic models for web search using clickthrough data[C]// Proceedings of the 22nd ACM International Confe-rence on Information and Knowledge Management, San Fran-cisco, Oct 27-Nov 1, 2013. New York: ACM, 2013: 2333-2338.
[16]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Confe-rence on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[17]	KE H, CHEN D, LI X, et al. Towards brain big data classifi-cation: epileptic EEG identification with a lightweight VGGNet on global MIC[J]. IEEE Access, 2018, 6: 14722-14733. DOI URL
[18]	GUO B, ZHANG C, LIU J, et al. Improving text classifi-cation with weighted word embeddings via a multi-channel TextCNN model[J]. Neurocomputing, 2019, 363: 366-374. DOI URL
[19]	KARIMI E, MAJIDI B, MANZURI M T. Relevant ques-tion answering in community based networks using deep LSTM neural networks[C]// Proceedings of the 2019 7th Ira-nian Joint Congress on Fuzzy and Intelligent Systems. Pisca-taway: IEEE, 2019: 1-5.
[20]	YANG M, TU W, QU Q, et al. Advanced community question answering by leveraging external knowledge and multitask learning[J]. Knowledge-Based Systems, 2019, 171: 106-119. DOI URL
[21]	DENG Y, LAM W, XIE Y, et al. Joint learning of answer selection and answer summary generation in community ques-tion answering[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2020: 7651-7658.
[22]	LIU J, YANG Y, LV S, et al. Attention-based BiGRU-CNN for Chinese question classification[J]. Journal of Ambient Inte-lligence and Humanized Computing, 2019. DOI: 10.1007/s12652-019-01344-9. DOI
[23]	ZHANG Y, LU W, OU W, et al. Chinese medical question ans-wer selection via hybrid models based on CNN and GRU[J]. Multimedia Tools and Applications, 2020, 79(21): 14751-14776. DOI URL
[24]	YUAN S, ZHANG Y, TANG J, et al. Expert finding in commu-nity question answering: a review[J]. Artificial Intelligence Review, 2020, 53(2): 843-874. DOI URL
[25]	FU J, LI Y, ZHANG Q, et al. Recurrent memory reasoning network for expert finding in community question answering[C]// Proceedings of the 13th International Conference on Web Search and Data Mining. New York: ACM, 2020: 187-195.

名称	数量
用户描述记录	1 931 654
问题描述记录	1 829 900
问题邀请记录	500 000
主题标签	100 000

名称	数量
用户描述记录	1 931 654
问题描述记录	1 829 900
问题邀请记录	500 000
主题标签	100 000

算法名称	AUC			ACC			Logloss
算法名称	30%	70%	100%	30%	70%	100%	30%	70%	100%
DeepFM	0.522 3	0.530 1	0.532 3	0.724 9	0.731 8	0.741 0	0.466 6	0.466 8	0.468 3
XDeepFM	0.574 1	0.576 0	0.584 7	0.754 0	0.766 0	0.788 6	0.462 9	0.461 8	0.460 7
CNN-DSSM	0.584 8	0.605 9	0.613 3	0.787 5	0.800 5	0.801 8	0.457 5	0.449 2	0.449 0
DSIERM-OS	0.628 8	0.638 5	0.645 6	0.801 2	0.809 0	0.811 2	0.451 0	0.439 1	0.437 1
DSIERM	0.630 2	0.647 6	0.649 1	0.809 6	0.810 2	0.811 6	0.439 1	0.431 7	0.431 0

算法名称	AUC			ACC			Logloss
算法名称	30%	70%	100%	30%	70%	100%	30%	70%	100%
DeepFM	0.522 3	0.530 1	0.532 3	0.724 9	0.731 8	0.741 0	0.466 6	0.466 8	0.468 3
XDeepFM	0.574 1	0.576 0	0.584 7	0.754 0	0.766 0	0.788 6	0.462 9	0.461 8	0.460 7
CNN-DSSM	0.584 8	0.605 9	0.613 3	0.787 5	0.800 5	0.801 8	0.457 5	0.449 2	0.449 0
DSIERM-OS	0.628 8	0.638 5	0.645 6	0.801 2	0.809 0	0.811 2	0.451 0	0.439 1	0.437 1
DSIERM	0.630 2	0.647 6	0.649 1	0.809 6	0.810 2	0.811 6	0.439 1	0.431 7	0.431 0

Expert Recommendation Algorithm Combining Attention and Recurrent Neural Network

结合注意力与循环神经网络的专家推荐算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 25

Related Articles 15

Recommended Articles

Metrics

[1]	LI Zhenqi, WANG Jing, JIA Ziyu, LIN Youfang. Attention-Based Multi-dimensional Feature Graph Convolutional Network for Motor Imagery Classification [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2050-2060.
[2]	ZHANG Xiangping, LIU Jianxun. Overview of Deep Learning-Based Code Representation and Its Applications [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2011-2029.
[3]	LI Dongmei, LUO Sisi, ZHANG Xiaoping, XU Fu. Review on Named Entity Recognition [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1954-1968.
[4]	REN Ning, FU Yan, WU Yanxia, LIANG Pengju, HAN Xi. Review of Research on Imbalance Problem in Deep Learning Applied to Object Detection [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1933-1953.
[5]	YANG Caidong, LI Chengyang, LI Zhongbo, XIE Yongqiang, SUN Fangwei, QI Jin. Review of Image Super-resolution Reconstruction Algorithms Based on Deep Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1990-2010.
[6]	ZENG Fanzhi, XU Luqian, ZHOU Yan, ZHOU Yuexia, LIAO Junwei. Review of Knowledge Tracing Model for Intelligent Education [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1742-1763.
[7]	YANG Zhiqiao, ZHANG Ying, WANG Xinjie, ZHANG Dongbo, WANG Yu. Application Research of Improved U-shaped Network in Detection of Retinopathy [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1877-1884.
[8]	AN Fengping, LI Xiaowei, CAO Xiang. Medical Image Classification Algorithm Based on Weight Initialization-Sliding Window CNN [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1885-1897.
[9]	LIU Yi, LI Mengmeng, ZHENG Qibin, QIN Wei, REN Xiaoguang. Survey on Video Object Tracking Algorithms [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1504-1515.
[10]	ZHAO Xiaoming, YANG Yijiao, ZHANG Shiqing. Survey of Deep Learning Based Multimodal Emotion Recognition [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1479-1503.
[11]	XIA Hongbin, XIAO Yifei, LIU Yuan. Long Text Generation Adversarial Network Model with Self-Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1603-1610.
[12]	LI Yunhuan, WEN Jiwei, PENG Li. High Frame Rate Light-Weight Siamese Network Target Tracking [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1405-1416.
[13]	SUN Fangwei, LI Chengyang, XIE Yongqiang, LI Zhongbo, YANG Caidong, QI Jin. Review of Deep Learning Applied to Occluded Object Detection [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1243-1259.
[14]	LIU Yafen, ZHENG Yifeng, JIANG Lingyi, LI Guohe, ZHANG Wenjie. Survey on Pseudo-Labeling Methods in Deep Semi-supervised Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1279-1290.
[15]	ZHANG Yancao, ZHAO Yuhai, SHI Lan. Multi-feature Based Link Prediction Algorithm Fusing Graph Attention [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1096-1106.