Journal of Frontiers of Computer Science and Technology ›› 2024, Vol. 18 ›› Issue (6): 1590-1599.DOI: 10.3778/j.issn.1673-9418.2308085

• Artificial Intelligence·Pattern Recognition • Previous Articles     Next Articles

Research on Knowledge Graph Entity Prediction Method of Multi-modal Curriculum Learning

XU Zhihong, HAO Xuemei, WANG Liqin, DONG Yongfeng, WANG Xu   

  1. 1. School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin 300401, China
    2. Hebei Key Laboratory of Big Data Computing, Tianjin 300401, China
    3. Hebei Engineering Research Center of Data-Driven Industrial Intelligence, Tianjin 300401, China
  • Online:2024-06-01 Published:2024-05-31

多模态课程学习知识图谱实体预测方法研究

许智宏,郝雪梅,王利琴,董永峰,王旭   

  1. 1. 河北工业大学 人工智能与数据科学学院,天津 300401
    2. 河北省大数据计算重点实验室,天津 300401
    3. 河北省数据驱动工业智能工程研究中心,天津 300401

Abstract: On the one hand, the existing knowledge graph entity prediction methods only use the neighborhood and graph structure information to enhance the node information, and ignore the multi-modal information outside the knowledge graph to enhance the knowledge graph information. On the other hand, when comparing positive and negative samples to train the model, the negative sample random ordering results in poor training effect, and there is no additional information to help the training process of negative samples. Therefore, a multi-modal curriculum learning knowledge graph entity prediction model (MMCL) is proposed. Firstly, multi-modal information is introduced into the knowledge graph to achieve information enhancement, and the multi-modal information fusion process is optimized using generative adversarial network (GAN). The samples generated by the generator enhance the knowledge graph information, and at the same time improve the discriminator??s ability to distinguish the truth and falsity of triples. Secondly, the course learning algorithm is used to sort the negative samples from easy to difficult according to the difficulty of the negative samples. By adding the sorted negative samples into the training process hierarchically through the pace function, it is more beneficial to playing the effect of negative samples in identifying the truth and falseness of triples, and at the same time, no label learning avoids the false-negative problem in the late training period. The discriminators share parameters with course learning training models to help improve the training effect of negative samples. Experiments are conducted on two datasets, FB15k-237 and WN18RR. The results show that compared with the baseline model, MMCL is significantly improved in mean reciprocal rank (MRR), Hits@1, Hits@3 and Hits@10. The validity and feasibility of the proposed model are verified.

Key words: curriculum learning, multi-modal, generative adversarial network (GAN), negative sample

摘要: 现有知识图谱实体预测方法一方面只利用邻域和图结构信息增强节点信息,忽略了知识图谱之外的多模态信息对于知识图谱信息的增强;另一方面正负样本对比训练模型时负样本随机排序导致训练效果不佳,且没有额外的信息帮助负样本的训练过程。为此,提出了一种多模态课程学习知识图谱实体预测模型(MMCL)。首先把多模态信息引入知识图谱实现信息增强,利用生成对抗网络(GAN)优化多模态信息融合过程,生成器生成的样本增强知识图谱信息,同时也提升鉴别器判别三元组真伪的能力;其次利用课程学习算法根据负样本的难易程度对负样本从易到难排序,通过步调函数分层次地把排序的负样本加入到训练过程中,更有利于发挥负样本鉴别三元组真伪的效果,同时无标签学习避免了训练后期假阴性问题;多模态信息融合互相优化的鉴别器与课程学习训练模型共享参数,帮助提升负样本的训练效果。在FB15k-237和WN18RR两个数据集上进行实验,结果表明,MMCL与基线模型相比,在平均倒数排名(MRR)、Hits@1、Hits@3以及Hits@10四个性能评价指标均有明显提升,验证了所提模型的有效性和可行性。

关键词: 课程学习, 多模态, 生成对抗网络(GAN), 负采样