Journal of Frontiers of Computer Science and Technology ›› 2023, Vol. 17 ›› Issue (1): 166-178.DOI: 10.3778/j.issn.1673-9418.2105001

• Graphics·Image • Previous Articles     Next Articles

Construction and Matching of Thermal Feature Descriptor for Image Retrieval

LIU Tianyu, JIA Di, LUO Shunli, WANG Kai   

  1. School of Electronic and Information Engineering, Liaoning Technical University, Huludao, Liaoning 125105, China
  • Online:2023-01-01 Published:2023-01-01



  1. 辽宁工程技术大学?电子与信息工程学院,辽宁 葫芦岛?125105

Abstract: Image matching technology based on deep learning has become a research hotspot in image retrieval tasks. Affected by the high similarity descriptors of different local regions, the mismatch problem reduces the retrieval accuracy of the existing methods. For this reason, a method of constructing and matching thermal feature descriptors for image retrieval is proposed. Firstly, the feature map of each layer is obtained through the semantic segmentation network. By observing the feature map, it is found that the last convolutional layer has the characteristics of stronger spatial and semantic information. For this feature, the gradient score is used to obtain the weight of each channel in the feature map. Through linear fusion, the channel dimension is weighted and summed and normalized, and the final heat map is obtained by bilinear interpolation. Secondly, the deep local features (DELF) algorithm is used to obtain the image depth feature descriptor. The category information and thermal value information obtained by the semantic segmentation network are used to construct a multi-dimensional composite depth feature descriptor, and the KD tree structure for this type of feature descriptor is given. Finally, based on the structure, it combines BBF (best bin first) and random sampling consensus algorithm to achieve feature matching. Experiments on the Oxford5K and Paris6K public datasets show that the proposed method is better in precision and time compared with the DELF and D2-Net algorithm. Compared with Fine-tuning CNN, DAME WEB and other methods, the retrieval accuracy is improved by nearly 2 percentage points. The method in this paper can better improve the efficiency and accuracy of image retrieval, and the experimental results verify the effectiveness of the method in this paper.

Key words: image retrieval, semantic segmentation network, deep feature descriptor, heat map, KD tree

摘要: 基于深度学习的图像匹配技术已成为图像检索任务的研究热点,受不同局部区域高相似性描述子影响,误匹配问题降低了现有方法的检索精度,为此提出一种面向图像检索的热力特征描述子构造与匹配方法。首先,通过语义分割网络获得每个卷积层的可视化特征图,针对最后一个卷积层具有更强空间信息和语义信息的特点,利用梯度得分获得特征图中每个通道的权重,通过线性融合的方式,在通道维度上加权求和并归一化,利用双线性插值获得最终的热力图。其次,采用深度学习方法(DELF)获得图像深度特征描述子,利用语义分割网络所获得的类别信息和热力值信息构造多维度复合热力特征描述子,给出针对这类特征描述子的KD树结构。最后,基于该结构结合BBF和随机抽样一致算法实现特征匹配。在Oxford5K和Paris6K数据集上进行实验,实验结果表明,与DELF和D2-Net算法相比,该方法在查准率及时间效率上都有所提高,与Fine-tuning CNN、DAME WEB等方法相比检索精度提高近2个百分点。该方法能够更好地提高图像检索效率和精度,实验结果验证了该方法的有效性。

关键词: 图像检索, 语义分割网络, 深度特征描述子, 热力图, KD树