Keyphrase Extraction Combining Word and Document Embeddings

doi:10.3778/j.issn.1673-9418.2003022

Abstract

Abstract:

With the increasing amount of text data in various application fields, how to quickly and accurately extract the main information has become the main task of keyphrase extraction. This paper proposes a novel method for keyphrase extraction based on word and document vectors. By calculating vector representation between word and document on the same dimensional vector space, the semantic similarity between word and document can be got, which can be used as the initial weight of each word node in the undirected graph. Then, this paper calculates the score of each word and candidate phrase using a semantic biased random walk strategy. Finally, the top[N] scored candidate phrases are selected as the final keyphrases. Experimental results on the public datasets show that the proposed algorithm outperforms the state-of-the-art keyphrase extraction methods in precision, recall, and F-measure. It can greatly improve the efficiency of automatic keyphrase extraction.

Key words: keyphrase extraction, graph sorting, word embedding, document embedding, semantic information

摘要：

各类应用领域的文本数据日益增多，如何从这些海量数据中迅速准确地提取核心内容，已成为关键词抽取的主要任务。提出一种基于词和文档嵌入的关键词抽取方法，通过计算单词与文档在同一维度上的向量表示，得出每个单词与文档之间的语义相似度，将其作为无向图中每个单词节点的初始权重。接着使用带语义偏向的随机游走策略，计算出每个单词以及候选词的分值。最后选取得分较高的前[N]个候选词作为最终关键词。在公开数据集上的实验结果表明，该算法在准确率、召回率、[F]值上均超过现有的主流关键词抽取方法，极大提高了关键词自动抽取的效率。

关键词: 关键词抽取, 图排序, 词嵌入, 文档嵌入, 语义信息

ZU Xian, XIE Fei, LIU Xiaojian. Keyphrase Extraction Combining Word and Document Embeddings[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(2): 294-304.

祖弦, 谢飞, 刘啸剑. 融合词和文档嵌入的关键词抽取算法[J]. 计算机科学与探索, 2021, 15(2): 294-304.

References

[1] KIM S N, MEDELYAN O, KAN M Y, et al. Automatic key-phrase extraction from scientific articles[J]. Language Resources and Evaluation, 2013, 47(3): 723-742.
[2] HASSAINE A, MECHETER S, JAOUA A. Text categori-zation using hyper rectangular keyword extraction: application to news articles classi?cation[C]//LNCS 9348: Proceedings of the 2015 International Conference on Relational and Alg-ebraic Methods in Computer Science, Portugal, Sep 28-Oct 1, 2015. Berlin, Heidelberg: Springer, 2015: 312-325.
[3] LIU X, SONG Y, LIU S, et al. Automatic taxonomy con-struction from keywords[C]//Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Beijing, Aug 23-27, 2012. New York: ACM, 2012: 1433-1441.
[4] HADDOUD M, ABDEDDA?M S. Accurate keyphrase extra-ction by discriminating overlapping phrases[J]. Journal of Information Science, 2014, 40(4): 488-500.
[5] STERCKX L, CARAGEA C, Demeester T, et al. Supervised keyphrase extraction as positive unlabeled learning[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 1924-1929.
[6] GOLLAPALLI S D, LI X L, YANG P. Incorporating expert knowledge into keyphrase extraction[C]//Proceedings of the 31st AAAI Conference on Atificial Intelligence, San Francisco, Feb 4-9, 2017. San Francisco: AAAI, 2017: 3180-3187.
[7] XIE F, WU X, ZHU X. Efficient sequential pattern mining with wildcards for keyphrase extraction[J]. Knowledge-Based Systems, 2017, 115: 27-39.
[8] ALZAIDY R, CARAGEA C, Giles C L. Bi-LSTM-CRF sequence labeling for keyphrase extraction from scholarly documents[C]//Proceedings of the 2019 World Wide Web Conference, San Francisco, May 13-17, 2019. New York: ACM, 2019: 2551-2557.
[9] SANTOSH T Y S S, SANYAL D K, Bhowmick P K, et al. DAKE： document-level attention for keyphrase extraction[J]. Advances in Information Retrieval, 2020: 392-401.
[10] EL-BELTAGY S R, RAFEA A. KP-Miner: participation in SemEval-2[C]//Proceedings of the 5th International Work-shop on Semantic Evaluation, Uppsala, Jul 15-16, 2010. Str-oudsburg: ACL, 2010: 190-193.
[11] EMU I H, AHMED A U, ISIAM M, et al. An efficient approach for keyphrase extraction from english document[J]. International Journal of Intelligent Systems and App-lications, 2017, 12(9): 59-66.
[12] MIHALCEA R, TARAU P. TextRank: bring order into texts[C]//Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, Barcelona, Jul 25-26, 2004. Stroudsburg: ACL, 2004: 404-411.
[13] WAN X, XIAO J. Single document keyphrase extraction using neighborhood knowledge[C]//Proceedings of the 23rd National Conference on Artificial Intelligence, Chicago, Jul 13-17, 2008. Menlo Park: AAAI, 2008: 855-860.
[14] BELLAACHIA A, AL-DHELAAN M. NE-Rank: a novel graph-based key phrase extraction in Twitter[C]//Procee-dings of the 2012 IEEE/WIC/ACM International Joint Con-ferences on Web Intelligence and Intelligent Agent Tech-nology, Macau, China, Dec 4-7, 2012. Piscataway: IEEE, 2012: 372-379.
[15] FLORESCU C, CARAGEA C. Positionrank: an unsupervised approach to keyphrase extraction from scholarly documents[C]//Proceedings of the 55th Annual Meeting of the Ass-ociation for Computational Linguistics, Vancouver, Jul 30-Aug 4, 2017. Stroudsburg: ACL, 2017: 1105-1115.
[16] YAN Y, TAN Q P, XIE Q Z, et al. A graph-based approach of automatic keyphrase extraction[J]. Procedia Computer Science, 2017, 107: 248-255.
[17] BISWAS S K, BORDOLOI M, SHREYA J. A graph based keyword extraction model using collective node weight[J]. Expert Systems with Applications, 2018, 97: 51-59.
[18] LIU Z Y, HUANG W Y, ZHENG Y B, et al. Automatic keyphrase extraction via topic decomposition[C]//Proceedings of the 2010 Conference on Empirical Methods in Natural Lang-uage Processing, Massachusetts, Oct 9-11, 2010. Stroudsburg: ACL, 2010: 366-376.
[19] STERCKX L, DEMEESTER T, DELEU J, et al. Topical word importance for fast keyphrase extraction[C]//Proceedings of the 24th International Conference on World Wide Web, Florence, May 18-22, 2015. New York: ACM, 2015: 121-122.
[20] TENEVA N, CHENG W. Salience rank: efficient keyphrase extraction with topic modeling[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Ling-uistics, Vancouver, Jul 30-Aug 4, 2017. Stroudsburg: ACL, 2017: 530-535.
[21] BOUGOUIN A, BOUDIN F, DAILLE B. TopicRank: graph-based topic ranking for keyphrase extraction[C]//Proceedings of the 6th International Joint Conference on Natural Language Processing, Nagoya, Oct 14-18, 2013. Stroudsburg: ACL, 2013: 543-551.
[22] BOUDIN F. Unsupervised keyphrase extraction with mul-tipartite graphs[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Com-putational Linguistics: Human Language Technologies, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 667-672.
[23] LI L, LIU J H, SUN Y Q, et al. Unsupervised keyword extraction from microblog posts via hashtags[J]. Journal of Web Engineering, 2018, 17: 93-120.
[24] MAHATA D, KURIAKOSE J, SHAH R R, et al. Key2Vec: automatic ranked keyphrase extraction from scientific articles using phrase embeddings[C]//Proceedings of the 2018 Con-ference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 634-639.
[25] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[C]//Pro-ceedings of the 2013 International Conference on Learning Representations, Scottsdale, May 2-4, 2013: 1-12.
[26] PENNINGTON J, SOCHER R, MANNING C. Glove: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stroudsburg: ACL, 2014: 1532-1543.
[27] PAGLIARDINI M, GUPTA P, JAGGI M. Unsupervised learning of sentence embeddings using compositional n-gram features[J]. arXiv:1703.02507, 2017.
[28] LE Q, MIKOLOV T. Distributed representations of sentences and documents[J]. arXiv:1405.4053, 2014.
[29] BENNANI-SMIRES K, MUSAT C, HOSSMANN A, et al. EmbedRank: unsupervised keyphrase extraction using sen-tence embeddings[J]. arXiv:1801.04470, 2018.
[30] WANG Z H, GUO Y. Sentence-ranking-enhanced keywords extraction from Chinese patents[J]. Journal of Information Science and Engineering, 2019, 35(3): 651-674.