关键词: 领域知识图谱, 预训练模型, 自注意力机制, 特征融合

Abstract: Entity relationship extraction aims to detect the relationship between entities and entity pairs from unstruc-tured text. It is an important step in constructing domain knowledge map. In view of the poor semantic expression ability of the existing extraction models and the low accuracy of overlapping triples extraction, this paper studies the joint extraction of entity relationships by integrating pre-trained model and attention, and divides the entity relation-ship extraction task into two tag modules. The head entity tagging module uses a pre-trained model to encode sen-tences. In order to further learn the internal characteristics of sentences, bi-directional long-short term memory and self-attention mechanism are used to form a feature enhancement layer. The binary classifier is used as the decoder of the model to mark the start and end positions of the head entity in the sentence. In order to deepen the relationship between the two marking modules, a feature fusion layer is set up before the tail entity marking task. The head entity features and sentence vectors are fused through convolutional neural networks (CNN) and attention mechanism. The relationship between entities is determined and the tail entity is marked through multiple identical and independent binary classifiers. A joint model based on pre-trained encoder and attention mechanism (JPEA) is constructed. Experimental results show that this method can significantly improve the extraction effect, and the performance of extraction tasks under different pre-trained models is compared, which further illustrates the superio-rity of the model.

Key words: domain knowledge graph, pre-trained model, self-attention mechanism, feature fusion