Abstract: Entity relationship extraction aims to detect the relationship between entities and entity pairs from unstruc-tured text. It is an important step in constructing domain knowledge map. In view of the poor semantic expression ability of the existing extraction models and the low accuracy of overlapping triples extraction, this paper studies the joint extraction of entity relationships by integrating pre-trained model and attention, and divides the entity relation-ship extraction task into two tag modules. The head entity tagging module uses a pre-trained model to encode sen-tences. In order to further learn the internal characteristics of sentences, bi-directional long-short term memory and self-attention mechanism are used to form a feature enhancement layer. The binary classifier is used as the decoder of the model to mark the start and end positions of the head entity in the sentence. In order to deepen the relationship between the two marking modules, a feature fusion layer is set up before the tail entity marking task. The head entity features and sentence vectors are fused through convolutional neural networks (CNN) and attention mechanism. The relationship between entities is determined and the tail entity is marked through multiple identical and independent binary classifiers. A joint model based on pre-trained encoder and attention mechanism (JPEA) is constructed. Experimental results show that this method can significantly improve the extraction effect, and the performance of extraction tasks under different pre-trained models is compared, which further illustrates the superio-rity of the model.

Key words: domain knowledge graph, pre-trained model, self-attention mechanism, feature fusion

摘要： 实体关系抽取旨在从无结构的文档中检测出实体和实体对的关系，是构建领域知识图谱的重要步骤。针对现有抽取模型语义表达能力差、重叠三元组抽取准确率低的情况，研究了融合预训练模型和注意力的实体关系联合抽取问题，将实体关系抽取任务分解为两个标记模块。头实体标记模块采用预训练模型对句子进行编码，为了进一步学习句子的内在特征，利用双向长短时记忆网络（BiLSTM）和自注意力机制组成特征加强层。采用二进制分类器作为模型的解码器，标记出头实体在句子中的起止位置。为了加深两个标记模块之间的联系，在尾实体标记任务前设置特征融合层，将头实体特征与句子向量通过卷积神经网络（CNN）和注意力机制进行特征融合，通过多个相同且独立的二进制分类器判定实体间关系并标记尾实体，构建出融合预训练模型和注意力的联合抽取模型（JPEA）。实验结果表明，该方法能显著提升抽取的效果，对比不同预训练模型下抽取任务的性能，进一步说明了模型的优越性。

关键词: 领域知识图谱, 预训练模型, 自注意力机制, 特征融合