在线图正则化非负矩阵分解跨模态哈希

doi:10.3778/j.issn.1673-9418.2105039

摘要/Abstract

摘要： 跨模态哈希具有存储成本低、查询速度快的特性，是一种有效的跨媒体检索方法，近年来得到了广泛的关注。然而，现有的跨模态哈希方法普遍基于批量学习模式来学习哈希函数，该模式不能处理大规模数据集，内存消耗大，训练流数据效率较低。为解决上述问题，提出了在线跨模态哈希方法，但大多数方法均关注于将不同模态的数据映射到一个公共的低维空间中，从而消除不同模态之间异构性，实现跨模态检索。而对于同一模态内数据之间的相关性没有充分挖掘利用，使得最终学习到的哈希编码检索精度不高。提出在线图正则化非负矩阵分解跨模态哈希（OGNMFH）检索方法，充分利用数据模态内的局部流形结构信息和数据的类别标签信息，同时保持数据模态间相似性和模态内相似性，得到判别性更高的哈希码。通过在三个经典数据集上进行大量实验，证明了该方法可有效提升在线哈希学习的检索精度。

关键词: 跨模态检索, 在线哈希, 矩阵分解, 离散优化

Abstract: Because of low storage cost and fast query speed, cross-modal hash is an effective cross-media retrieval method, which has been widely studied in recent years. However, most of the existing cross-modal hashing methods are based on the batch learning model, which does not effectively handle large datasets, consumes a lot of memory, and is inefficient in training stream data. Online learning can be used for cross-modal hashing to solve the above problems. However, most online cross-modal hashing methods mainly focus on mapping the data of different modes into a common low-dimensional space, so as to eliminate the heterogeneity between different modes and achieve cross-modal retrieval. However, the correlation between the data in the same mode is not fully exploited, and the final hashing code retrieval accuracy is not high. In this paper, online graph regularized non-negative matrix factorization cross-modal hashing (OGNMFH) retrieval method is proposed. By making full use of the local manifold structure information in the data mode and the category label information of the data, the inter-modal similarity and intra-modal similarity of the data are kept, and higher discriminant hash codes are obtained. A large number of experimental results on three classical datasets demonstrate that the OGNMFH method can improve retrieval accuracy in online hash learning.

Key words: cross-modal retrieval, online hashing, matrix factorization, discrete optimization

罗雪梅, 郑海红, 安亚强, 王笛. 在线图正则化非负矩阵分解跨模态哈希[J]. 计算机科学与探索, 2023, 17(3): 678-686.

LUO Xuemei, ZHENG Haihong, AN Yaqiang, WANG Di. Online Graph Regularized Non-negative Matrix Factorization Cross-Modal Hashing[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(3): 678-686.

参考文献

[1] 欧卫华, 刘彬, 周永辉, 等. 跨模态检索研究综述[J]. 贵州师范大学学报(自然科学版), 2018, 138(2): 118-124.
OU W H, LIU B, ZHOU Y H, et al. Survey on the cross-modal retrieval research[J]. Journal of Guizhou Normal University (Natural Sciences), 2018, 138(2): 118-124.
[2] NIE X S, WANG B W, LI J J, et al. Deep multiscale fusion hashing for cross-modal retrieval[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(1): 401-410.
[3] CHENG M M, JING L P, NG M K. Robust unsupervised cross-modal hashing for multimedia retrieval[J]. ACM Transactions on Information Systems, 2020, 38(3): 1-25.
[4] MANDAL D, RAO P, BISWAS S. Semi-supervised cross-modal retrieval with label prediction[J]. IEEE Transactions on Multimedia, 2020, 22(9): 2345-2353.
[5] MANDAL D, CHAUDHURY K N, BISWAS S. Generalized semantic preserving hashing for cross-modal retrieval[J]. IEEE Transactions on Image Processing, 2019, 28(1): 102-112.
[6] XIE L, SHEN J, ZHU L. Online cross-modal hashing for web image retrieval[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, Feb 12-17, 2016. Menlo Park: AAAI, 2016: 294-300.
[7] XIE L, SHEN J L, HAN J G, et al. Dynamic multi-view hashing for online image retrieval[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Aug 19-25, 2017: 3133-3139.
[8] YAO T, WANG G, YAN L, et al. Online latent semantic hashing for cross-media retrieval[J]. Pattern Recognition, 2019, 89: 1-11.
[9] LEE D D, SEUNG H S. Learning the parts of objects by non-negative matrix factorization[J]. Nature, 1999, 401(21): 788-791.
[10] CAI D, HE X, HAN J, et al. Graph regularized nonnegative matrix factorization for data representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(8): 1548-1560.
[11] LIU F D, GUAN N Y, TANG Y H. Online graph regularized non-negative matrix factorization for streamming data[C]//Proceedings of the 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics, Wuhan, Oct 18-19, 2014. Piscataway: IEEE, 2014: 191-196.
[12] HUISKES M J, LEW M S. The MIR Flickr retrieval evaluation[C]//Proceedings of the 1st ACM SIGMM International Conference on Multimedia Information Retrieval, Vancouver, Oct 30-31, 2008. New York: ACM, 2008: 39-43.
[13] CHUA T S, TANG J H, HONG R, et al. NUS-WIDE: a real-world web image database from National University of Singapore[C]//Proceedings of the 8th ACM International Conference on Image and Video Retrieval, Santorini Island, Jul 8-10, 2009. New York: ACM, 2009: 1-9.
[14] LIN T Y, MAIRE M, BELONGIE S J, et al. Microsoft COCO: common objects in context[C]//LNCS 8693: Proceedings of the 13th European Conference on Computer Vision, Zurich, Sep 6-12, 2014. Cham: Springer, 2014: 740-755.
[15] WANG D, GAO X B, WANG X M, et al. Label consistent matrix factorization hashing for large-scale cross-modal similarity search[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(10): 2466-2479.