Journal of Frontiers of Computer Science and Technology ›› 2021, Vol. 15 ›› Issue (3): 506-514.DOI: 10.3778/j.issn.1673-9418.2004068

• Artificial Intelligence • Previous Articles     Next Articles

Cross-Modal Retrieval by Class Information and Listwise Ranking

LIU Yuping, GE Hong, ZENG Yibin   

  1. School of Computer Science, South China Normal University, Guangzhou 510631, China
  • Online:2021-03-01 Published:2021-03-05

利用类别信息和列表排序的跨模态检索

刘雨萍葛红曾奕斌   

  1. 华南师范大学 计算机学院,广州 510631

Abstract:

Cross-modal retrieval has attracted significant attention due to the increasing use of multi-modal data. A major challenge for cross-modal retrieval is the modal gap. To cope with the heterogeneity, common subspace learning method is proposed. However, most of them mainly focus on relevant or irrelevant information, and do not consider the relevant and irrelevant information simultaneously. In addition, there are many pairwise methods for cross-modal retrieval, but they do not consider the internal dependencies between the doc pairs corresponding to the same query and do not make full use of the structure between the samples. To take full account of the intra-class and inter-class relationships between samples, the cross-modal retrieval by listwise ranking and class information (C2MLR2) is proposed, which maximizes the similarity of positive samples to the anchor and meanwhile minimizes the similarity of the negative samples to the anchor via listwise ranking. Experimental results verify the effectiveness of the algorithm in cross-modal retrieval.

Key words: relevant information, irrelevant information, intra-class, inter-class, listwise ranking

摘要:

随着越来越多多模态数据的出现,跨模态检索引起了广泛的关注。跨模态检索面临一大挑战为模态鸿沟,为了解决数据的异构性问题,公共子空间学习的方法被提出。然而,大部分方法仅仅是单独考虑了样本之间的相关联信息或不相关信息,而没有同时考虑样本间的相关信息和不相关信息。除此之外,大部分方法对于样本之间相似度的比较,使用的是基于文档对的排序比较,其没有充分考虑样本之间的类内依赖性和类间样本的结构差异性。基于此,提出了一种同时而不是单独考虑样本间的类内关系和类间关系的基于列表排序的跨模态检索方法,其通过列表排序最大化锚点与正样本之间的相似性,同时最小化锚点和负样本间的相似性。实验结果验证了该算法在跨模态检索中的有效性。

关键词: 相关信息, 不相关信息, 类内, 类间, 列表排序