Distant Supervision Relation Extraction Combining Attention Mechanism and Ontology

doi:10.3778/j.issn.1673-9418.1909066

Abstract

Abstract:

Relational extraction extracts relationships from unstructured text and outputs them in a structured form. In order to improve the extraction accuracy and reduce the dependence on manual annotation, this paper proposes a distant supervision relationship extraction model based on attention mechanism and ontology, attention piecewise convolutional neural networks with ontology restriction (APCNNs+OR). The model is divided into feature engineering extraction module, classifier module and ontology restriction layer. In the classifier module, this paper introduces and improves the instance-level attention mechanism to learn the weight of each sentence in the data bag better, effectively reducing the noise interference introduced by the distant supervision hypothesis and the word information interference between the two entities in the sentence. In the ontology restriction layer, the extraction result is constrained by introducing the domain ontology, which improves the accuracy of the relationship extraction. The experiment results of SemMed and GoldStandard corpus show that the model can effectively reduce the noise interference of the wrong label and has better relation extraction performance than the existing models.

Key words: relation extraction, ontology, distant supervision, attention mechanism

摘要：

关系抽取是从非结构化的文本中抽取关系，并以结构化的形式输出。为了提高抽取准确性并降低对人工标注的依赖，提出了基于注意力机制和本体的远程监督关系抽取模型（APCNNs+OR）。该模型分为特征工程提取模块、分类器模块、本体约束层。在分类器模块中，引入并改进了实例级注意力机制，更好地学习数据袋中每个句子的权重，有效地降低了远程监督假设引入的噪声干扰及句子中实体间的词语信息干扰。在本体约束层，通过引入领域本体对抽取结果进行约束，提高了抽取关系的准确性。SemMed和GoldStandard语料实验结果表明，该模型可有效降低错误标签的噪声干扰，比现有模型具有更好的关系抽取性能。

关键词: 关系抽取, 本体, 远程监督, 注意力机制

LI Yanjuan, ZANG Mingzhe, LIU Xiaoyan, LIU Yang, GUO Maozu. Distant Supervision Relation Extraction Combining Attention Mechanism and Ontology[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(9): 1554-1562.

李艳娟，臧明哲，刘晓燕，刘扬，郭茂祖. 结合注意力机制和本体的远程监督关系抽取[J]. 计算机科学与探索, 2020, 14(9): 1554-1562.

References

[1] Wang Y, Wang L, Rastegar-Mojarad M, et al. Clinical infor-mation extraction applications: a literature review[J]. Journal of Biomedical Informatics, 2018, 77: 34-49.
[2] Rindflesch T C, Fiszman M. The interaction of domain know-ledge and linguistic structure in natural language processing: interpreting hypernymic propositions in biomedical text[J]. Journal of Biomedical Informatics, 2003, 36(6): 462-477.
[3] Mazen A, Mahmood M K, Susan S. Linked open data-based framework for automatic biomedical ontology generation[J]. BMC Bioinformatics, 2018, 19(1): 319.
[4] Miao Z H, Ma C H, Gao Z Y, et al. Hidden feature extraction for unstructured agricultural environment based on supervised kernel locally linear embedding modeling[J]. Advances in Manufacturing, 2018, 6(4): 409-418.
[5] Panyam N C, Verspoor K, Cohn T, et al. Exploiting graph kernels for high performance biomedical relation extraction[J]. Journal of Biomedical Semantics, 2018, 9(1): 7.
[6] Craven M, Kumlien J. Constructing biological knowledge bases by extracting information from text sources[C]//Pro-ceedings of the 7th International Conference on Intelligent Systems for Molecular Biology, Heidelberg, Aug 6-10, 1999. Menlo Park: AAAI, 1999: 77-86.
[7] Mintz M, Bills S, Snow R, et al. Distant supervision for rela-tion extraction without labeled data[C]//Proceedings of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, Singapore, Aug 2-7, 2009. Stroudsburg: ACL, 2009: 1003-1011.
[8] Pan Y, Yishabay B, Yang J, et al. Distant supervised personal relation extraction using Chinese online resource[J]. Journal of Chinese Computer Systems, 2015, 36(4): 701-706.潘云, 布勒布丽汗·伊沙巴依, 杨静, 等. 利用中文在线资源的远程监督人物关系抽取[J]. 小型微型计算机系统, 2015, 36(4): 701-706.
[9] Xue L J, Xi M L, Wang M J, et al. Entity relation extraction based on rule inference engine[J]. Journal of Frontiers of Computer Science and Technology, 2016, 10(9): 1310-1319. 薛丽娟, 席梦隆, 王梦婕, 等. 基于规则推理引擎的实体关系抽取研究[J]. 计算机科学与探索, 2016, 10(9): 1310-1319.
[10] Socher R, Huval B, Manning C D, et al. Semantic composi-tionality through recursive matrix-vector spaces[C]//Procee-dings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jeju Island, Jul 12-14, 2012. Stroudsburg: ACL, 2012: 1201-1211.
[11] Zeng D J, Liu K, Lai S W, et al. Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics, Dublin, Aug 23-29, 2014. Stroudsburg: ACL, 2014: 2335-2344.
[12] Zeng D J, Liu K, Chen Y B, et al. Distant supervision for relation extraction via piecewise convolutional neural net-works[C]//Proceedings of the 2015 Conference on Empi-rical Methods in Natural Language Processing, Lisbon, Sep 17-21, 2015. Stroudsburg: ACL, 2015: 1753-1762.
[13] Lin Y K, Shen S Q, Liu Z Y, et al. Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the ACL, Berlin, Aug 7-12, 2016. Stroudsburg: ACL, 2016: 2124-2133.
[14] Zhou P, Xu J, Qi Z, et al. Distant supervision for relation extraction with hierarchical selective attention[J]. Neural Networks, 2018, 108: 240-247.
[15] Ji G L, Liu K, He S Z, et al. Distant supervision for relation extraction with sentence-level attention and entity descrip-tions[C]//Proceedings of the 31st AAAI Conference on Arti-ficial Intelligence, San Francisco, Feb 4-9, 2017. Menlo Park: AAAI, 2017: 3060-3066.
[16] Che J L, Tang L W, Deng S J, et al. Distant supervision Chinese relation extraction based on dual attention mecha-nism[J]. Computer Engineering and Applications, 2019, 55(20): 107-113. 车金立, 唐力伟, 邓士杰, 等. 基于双重注意力机制的远程监督中文关系抽取[J]. 计算机工程与应用, 2019, 55(20): 107-113.
[17] Collobert R, Weston J, Bottou L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12: 2493-2537.
[18] Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in vector space[J]. arXiv:1301.3781, 2013.
[19] Liu J, Li T, Xie P, et al. Urban big data fusion based on deep learning: an overview[J]. Information Fusion, 2020, 53: 123-133.
[20] Luong M T, Pham H, Manning C D. Effective approaches to attention-based neural machine translation[J]. arXiv:1508. 04025, 2015.
[21] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
[22] Kilicoglu H, Shin D, Fiszman M, et al. SemMedDB: a PubMed-scale repository of biomedical semantic predications[J]. Bioin-formatics, 2012, 28(23): 3158-3160.
[23] Aronson A R. Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program[C]//Proceedings of the American Medical Informatics Association Annual Symposium, Washington, Nov 3-7, 2001: 17-21.
[24] Kilicoglu H, Rosemblat G, Fiszman M, et al. Constructing a semantic predication gold standard from the biomedical literature[J]. BMC Bioinformatics, 2011, 12(1): 486.