Recommendation Method Integrating Review Text Hierarchical Attention with Time Information

doi:10.3778/j.issn.1673-9418.2009035

Abstract

Abstract:

The data sparsity problem caused by information overload restricts the recommendation performance of the matrix factorization model based on scoring data. The recommendation model integrated with reviews text can effectively alleviate the sparsity of scoring data. The current recommendation system uses reviews text to model users and items, most of them use users' reviews text on items as the data sources, and ignore the impacts of time information on user and item attributes. In response to this problem, a recommendation method which integrates review text hierarchical attention with time information (RHATR) is proposed. This method can fully mine the potential semantic information of review text and model the dynamic changes of user perferences and item features. This method mines effective information such as sentiment words and keywords in a single review text, learns user and item representation by applying word level attention to a single review text, applies review level attention to extracting effective reviews respectively for the user review set and item review set with temporal factor and further learns the dynamic representation of user preferences and item features. Finally, the user and item representation learned from the review text, and ID based item and user embedding as the final features, it captures the potential factors of each user and item. The experimental results show that the proposed method has a better effect in root mean square error (RMSE) on Amazon and Yelp datasets than the current baseline method.

Key words: recommendation system, review text, hierarchical attention, time information

摘要：

信息过载造成的数据稀疏性问题制约着基于评分数据的矩阵分解模型的推荐性能，融合评论文本的推荐模型能够有效缓解评分数据稀疏性。当前的推荐系统利用评论文本为用户和项目建模时，大多仅将用户对项目的评论作为数据来源，而忽视了时间信息对用户和项目属性的影响。针对此问题，提出了一种融合短文本层级注意力和时间信息的推荐方法（RHATR），该方法能够充分地挖掘评论文本潜在的语义信息，并为用户偏好和项目特征的动态变化进行建模。通过对单条评论文本应用单词级注意力，挖掘单条评论文本中情感词和关键词等有效信息，学习用户和项目表示；对含有时间因素的用户评论集和项目评论集分别应用评论级注意力，提取有效的评论，进一步学习用户偏好和项目特征动态表示。将从评论文本中学到的用户和项目表示以及基于ID的项目和用户嵌入作为最终特征，来捕获各用户和项目的潜在因素。实验结果表明，提出的方法相对于当前基线方法在Amazon和Yelp数据集上的均方根误差（RMSE）取得了较好的效果。

关键词: 推荐系统, 评论文本, 层级注意力, 时间信息

XING Changzheng, GUO Yalan, ZHANG Quangui, ZHAO Hongbao. Recommendation Method Integrating Review Text Hierarchical Attention with Time Information[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(11): 2222-2232.

邢长征，郭亚兰，张全贵，赵宏宝. 融合短文本层级注意力和时间信息的推荐方法[J]. 计算机科学与探索, 2021, 15(11): 2222-2232.

References

[1] TAY Y, LUU A T, HUI S C. Multi-pointer co-attention net-works for recommendation[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, London, Aug 19-23, 2018. New York: ACM, 2018: 2309-2318.
[2] KOREN Y, BELL R M, VOLINSKY C. Matrix factorization techniques for recommender systems[J]. IEEE Computer, 2009, 42(8): 30-37.
[3] KOREN Y. Factorization meets the neighborhood: a multi-faceted collaborative filtering model[C]//Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, Aug 24-27, 2008. New York: ACM, 2008: 426-434.
[4] LEE D D, SEUNG H S. Algorithms for non-negative matrix factorization[C]//Proceedings of the Advances in Neural Infor-mation Processing Systems, Denver, Dec 1, 2000. Red Hook: Curran Associates, 2000: 556-562.
[5] MNIH A, SALAKHUTDINOV R R. Probabilistic matrix factorization[C]//Proceedings of the 21st Annual Conference on Neural Information Processing Systems, Vancouver, Dec 3-6, 2007. Red Hook: Curran Associates, 2008: 1257-1264.
[6] HUANG L W, JIANG B T, LV S Y, et al. Survey on deep learning based recommender systems[J]. Chinese Journal of Computers, 2018, 41(7): 1619-1647.
黄立威, 江碧涛, 吕守业, 等. 基于深度学习的推荐系统研究综述[J]. 计算机学报, 2018, 41(7): 1619-1647.
[7] ZHENG L, NOROOZI V, YU P S. Joint deep modeling of users and items using reviews for recommendation[C]//Proceedings of the 10th ACM International Conference on Web Search and Data Mining, Cambridge, Feb 6-10, 2017. New York: ACM, 2017: 425-434.
[8] CHEN C, ZHANG M, LIU Y Q, et al. Neural attentional rating regression with review-level explanations[C]//Proceedings of the 2018 World Wide Web Conference, Lyon, Apr 23-27, 2018. New York: ACM, 2018: 1583-1592.
[9] SEO S, HUANG J, YANG H, et al. Interpretable convolutional neural networks with dual local and global attention for review rating prediction[C]//Proceedings of the 11th ACM Conference on Recommender Systems, Como, Aug 27-31, 2017. New York: ACM, 2017: 297-305.
[10] WANG Q Q, LI S, CHEN G. Word-driven and context-aware review modeling for recommendation[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management, Torino, Oct 22-26, 2018. New York: ACM, 2018: 1859-1862.
[11] BAO Y, FANG H, ZHANG J. TopicMF: simultaneously ex-ploiting ratings and reviews for recommendation[C]//Pro-ceedings of the 28th AAAI Conference on Artificial Intellig-ence, Québec, Jul 27-31, 2014. Menlo Park: AAAI, 2014: 2-8.
[12] LI P J, WANG Z H, REN Z C, et al. Neural rating regression with abstractive tips generation for recommendation[C]//Proceedings of the 40th International ACM SIGIR Confer-ence on Research and Development in Information Retrieval, Shinjuku, Aug 7-11, 2017. New York: ACM, 2017: 345-354.
[13] RICCI F, ROKACH L, SHAPIRA B. Introduction to recom-mender systems handbook[M]//Recommender Systems Hand-book. Berlin, Heidelberg: Springer, 2011.
[14] SU X Y, KHOSHGOFTAAR T M. A survey of collaborative filtering techniques[J]. Advances in Artificial Intelligence, 2009: 421425.
[15] LI T, ZHANG R F, GUO K H. Incremental collaborative filtering recommendation method for personalized websites[J]. Computer Engineering and Applications, 2019, 55(4): 225-232.
李婷, 张瑞芳, 郭克华. 面向个性化网站的增量协同过滤推荐方法[J]. 计算机工程与应用, 2019, 55(4): 225-232.
[16] WU Y, DUBOIS C, ZHENG A X, et al. Collaborative denoi-sing auto-encoders for top-n recommender systems[C]//Pro-ceedings of the 9th ACM International Conference on Web Search and Data Mining, San Francisco, Feb 22-25, 2016. New York: ACM, 2016: 153-162.
[17] LI S, KAWALE J, FU Y. Deep collaborative filtering via marginalized denoising auto-encoder[C]//Proceedings of the 24th ACM International Conference on Information and Knowledge Management, Melbourne, Oct 19-23, 2015. New York: ACM, 2015: 811-820.
[18] MCAULEY J J, LESKOVEC J. Hidden factors and hidden topics: understanding rating dimensions with review text[C]//Proceedings of the 7th ACM Conference on Recommender Systems, Hong Kong, China, Oct 12-16, 2013. New York: ACM, 2013: 165-172.
[19] CATHERINE R, COHEN W W. TransNets: learning to transform for recommendation[C]//Proceedings of the 11th ACM Conference on Recommender Systems, Como, Aug 27-31, 2017. New York: ACM, 2017: 288-296.
[20] ZHANG W, YUAN Q, HAN J W, et al. Collaborative multi-level embedding learning from reviews for rating prediction[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, Jul 9-15, 2016. Menlo Park: AAAI, 2016: 2986-2992.
[21] SEO S, HUANG J, YANG H, et al. Representation learning of users and items for review rating prediction using attention-based convolutional neural network[C]//Proceedings of the SDM Workshop on Machine Learning Methods for Recom-mender Systems, Houston, Apr 27-29, 2017. Philadelphia: SIAM, 2017: 1-8.
[22] TAN Y Z, ZHANG M, LIU Y Q, et al. Rating-boosted latent topics: understanding users and items with ratings and reviews[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, Jul 9-15, 2016. Menlo Park: AAAI, 2016: 2640-2646.
[23] ZHANG Z P, SHEN X Y. Research on user behavior recom-mendation method based on deep learning[J]. Computer Engineering and Applications, 2019, 55(4): 142-147.
张祖平, 沈晓阳. 基于深度学习的用户行为推荐方法研究[J]. 计算机工程与应用, 2019, 55(4): 142-147.
[24] XING C Z, ZHAO H B, ZHANG Q G, et al. Reviews text hierarchical attention and outer product for recommenda-tion method[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(6): 947-957.
邢长征, 赵宏宝, 张全贵, 等. 融合评论文本层级注意力和外积的推荐方法[J]. 计算机科学与探索, 2020, 14(6): 947-957.
[25] DING Y, LI X. Time weight collaborative filtering[C]//Proceedings of the 2005 ACM CIKM International Confer-ence on Information and Knowledge Management, Bremen, Oct 31-Nov 5, 2005. New York: ACM, 2005: 485-492.
[26] KOREN Y. Collaborative filtering with temporal dynamics[C]//Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, Jun 28-Jul 1, 2009. New York: ACM, 2009: 447-456.
[27] WANG S Q, LI X X, SUN F Z, et al. Survey of research on personalized news recommendation techniques[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(1): 18-29.
王少卿, 李鑫鑫, 孙福振, 等. 个性化新闻推荐技术研究综述[J]. 计算机科学与探索, 2020, 14(1): 18-29.
[28] ZHANG X W, HE K Q, WANG J, et al. Web service recommendation based on watchlist via temporal and tag preference fusion[C]//Proceedings of the 2014 IEEE Inter-national Conference on Web Services, Anchorage, Jun 27-Jul 2, 2014. Washington: IEEE Computer Society, 2014: 281-288.
[29] HE X N, DU X Y, WANG X, et al. Outer product-based neural collaborative filtering[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Jul 13-19, 2018. Menlo Park: AAAI, 2018: 2227-2233.
[30] KIM D H, PARK C, OH J, et al. Convolutional matrix factorization for document context-aware recommendation[C]//Proceedings of the 10th ACM Conference on Recom-mender Systems, Boston, Sep 15-19, 2016. New York: ACM, 2016: 233-240.