[1]滕少华,黄文彪,张 巍,等.标签与样本双语义增强的跨模态检索[J].江西师范大学学报(自然科学版),2023,(03):296-306.[doi:10.16357/j.cnki.issn1000-5862.2023.03.10]
　TENG Shaohua,HUANG Wenbiao,ZHANG Wei,et al.The Cross-Modal Hash with Tag and Sample Semantic Enhancements[J].Journal of Jiangxi Normal University:Natural Science Edition,2023,(03):296-306.[doi:10.16357/j.cnki.issn1000-5862.2023.03.10]
点击复制

标签与样本双语义增强的跨模态检索()
分享到：

《江西师范大学学报》（自然科学版）[ISSN:1006-6977/CN:61-1281/TN]

卷:
期数:
2023年03期
页码:
296-306
栏目:
出版日期:
2023-05-25

文章信息/Info

Title:
The Cross-Modal Hash with Tag and Sample Semantic Enhancements
文章编号:
1000-5862(2023)03-0296-11
作者:
滕少华1; 黄文彪1; 张 巍1; 滕璐瑶2
(1.广东工业大学计算机学院,广东 广州 510006; 2.广州番禺职业技术学院信息工程学院,广东 广州 511483)
Author(s):
TENG Shaohua1;  HUANG Wenbiao1;  ZHANG Wei1;  TENG Luyao2
(1.School of Computer Science, Guangdong University of Technology, Guangzhou Guangdong 510006, China; 2.School of Information Engineering, Guangzhou Panyu Polytechnic, Guangzhou Guangdong 511483,China)
关键词:
标签与样本双语义增强;  跨模态检索;  标签语义
Keywords:
tag and sample semantic enhancements;  cross-modal retrieval;  tag semantics
分类号:
TP 311
DOI:
10.16357/j.cnki.issn1000-5862.2023.03.10
文献标志码:
A
摘要:
针对目前大多数跨模态哈希检索方法无法捕获多标签信息和特征语义更深层的语义关系信息问题,该文提出了一种标签与样本双语义增强的跨模态检索框架.首先,该框架将不同模态的高维数据映射到低维共享特征语义空间中,进行样本语义学习; 其次,引入松弛变量到标签语义制约的哈希码学习函数中,通过最小化标签成对距离强化样本语义相似性哈希码学习,这样既保持了跨模态对应样本语义的关系,强化了哈希码的标签语义学习,又解决了实对称矩阵的求解及算法的收敛性问题; 再次,进一步应用样本特征语义和标签语义增强哈希码的语义学习; 最后,在3个常用的数据集上的实验结果表明该方法优于目前的方法.
Abstract:
Aiming at the problem that most cross-modal hash methods cannot capture the multi-tag information and the deeper semantic relationship information of feature semantics,a cross-modal retrieval framework with bilingual enhancement of tag and sample is proposed.The framework first decomposes different high-dimensional modal data into low-dimensional shared feature semantic space.Secondly,the hash code learning function that relaxes the variables into the tag semantic constraints is introduced to strengthen the sample semantic similarity hash code learning by minimizing the tag pair distance,which not only maintains the relationship between the cross-modal corresponding sample semantics,strengthens the tag semantic learning of the hash code,but also solves the problem of solving the real symmetric matrix and the convergence of the algorithm.Thirdly, further apply sample feature semantics and tag semantics to enhance the semantic learning of hash codes.Finally,the experimental results on three commonly used data sets show that this method is superior to the current advanced methods.

参考文献/References:

[1] TENG Luyao,TANG Feiyi,ZHENG Zefeng,et al.Kernel based sparse representation learning with global and local low-rank constrain [EB/OL].[2022-11-01].https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9989445.
[2] FANG Xiaozhao,HAN Na,ZHOU Guoxu,et al.Dynamic double classifiers approximation for cross-domain recognition [J].IEEE Transactions on Cybernetics,2022,52(4):2618-2629.
[3] FANG Xiaozhao,JIANG Kaihang,HAN Na,et al.Average approximate hashing-based double projections learning for cross-modal retrieval [J].IEEE Transactions on Cybernetics,2021,52(11):11780-11793.
[4] CHEN Zhenduo,LI Chuanxiang,LUO Xin, et al.SCRATCH:a scalable discrete matrix factorization hashing framework for cross-modal retrieval [J].IEEE Transactions on Circuits and Systems for Video Technology,2020,30(7):2262-2275.
[5] DING Guiguang,GUO Yuchen,ZHOU Jile.Collective matrix factorization hashing for multimodal data [EB/OL].[2022-11-01].https://ieeexplore.ieee.org/document/6909664.
[6] LIN Zijia,DING Guiguang,HU Mingqing,et al.Semantics-preserving hashing for cross-view retrieval [EB/OL].[2022-03-11].https://ieeexplore.ieee.org/document/7299011.
[7] LIU Hong,JI Rongrong,WU Yongjian,et al.Cross-modality binary code learning via fusion similarity hashing [EB/OL].[2022-03-11].https://ieeexplore.ieee.org/document/8100155/.
[8] LIU Xin,HU Zhikai,LING Haibin,et al.MTFH: a matrix tri-factorization hashing framework for efficient cross-modal retrieval [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,43(3):964- 981.
[9] QIN Jianyang,FEI Lunke,TENG Shaohua, et al.Discrete semantic matrix factorization hashing for cross-modal retrieval[EB/OL].[2022-03-15].https://ieeexplore.ieee.org/document/9413037/.
[10] ZHOU Jile, DING Guiguang, GUO Yuchen.Latent semantic sparse hashing for cross-modal similarity search[EB/OL].[2022-04-09].https://dl.acm.org/doi/10.1145/2600428.2609610.
[11] WANG Lu,YANG Jie,ZAREAPOOR M,et al.Cluster-wise unsupervised hashing for cross-modal similarity search [J].Pattern Recognition, 2021,111(5):107732.
[12] JIN Sheng, YAO Hongxun, ZHOU Qin, et al.Unsupervised discrete hashing with affinity similarity [J]. IEEE Transactions on Image Processing, 2021, 30:6130-6141.
[13] WANG Di, WANG Quan, HE Lihuo, et al. Joint and individual matrix factorization hashing for large-scale cross-modal retrieval[J].Pattern Recognition,2020,107:107479.
[14] TANG Jun, WANG Ke, SHAO Ling. Supervised matrix factorization hashing for cross-modal retrieval [J]. IEEE Transactions on Image Processing, 2016, 25(7):3157-3166.
[15] QIN Jianyang, FEI Lunke, ZHU Jian, et al. Scalable discriminative discrete hashing for large-scale cross-modal retrieval[EB/OL].[2022-04-19].https://ieeexplore.ieee.org/document/9413871.
[16] WANG Yongxin, LUO Xin, NIE Liqiang, et al. BATCH: a scalable asymmetric discrete cross-modal hashing [J]. IEEE Transactions on Knowledge and Data Engineering, 2021, 33(11):3507-3519.
[17] WU Fei, WU Zhiyong, FENG Yujian, et al. Supervised discrete matrix factorization hashing for cross-modal retrieval [EB/OL] [2022-11-15].https://ieeexplore.ieee.org/document/8691389.
[18] XU Xing, SHEN Fumin, YANG Yang, et al. Learning discriminative binary codes for large-scale cross-modal retrieval [J]. IEEE Transactions on Image Processing, 2017, 26(5):2494-2507.
[19] ZHANG Dongqing, LI Wujun. Large-scale supervised multimodal hashing with semantic correlation maximization [EB/OL].[2022-11-15]. https://dl.acm.org/doi/10.5555/2892753.2892854.
[20] ZHANG Pengfei, LI Chuanxiang, LIU Mengyuan, et al. Semi-relaxation supervised hashing for cross-modal retrieval [EB/OL].[2022-11-15]. https://doi.org/10.1145/3123266.3123320.
[21] YAO Tao, YAN Lianshan, MA Yilan, et al. Fast discrete cross-modal hashing with semantic consistency [J]. Neural Networks, 2020, 125(8):142-152.
[22] WANG Di, GAO Xinbo, WANG Xiumei, et al. Label consistent matrix factorization hashing for large-scale cross-modal similarity search [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,41(10):2466-2479.
[23] WANG Song, ZHAO Huan, NAI Kei. Learning a maximized shared latent factor for cross-modal hashing[J].Knowledge-Based Systems,2021,228(9):107252.
[24] ZHENG Chaoqun, ZHU Lei, LU Xu, et al. Fast discrete collaborative multi-modal hashing for large-scale multimedia retrieval [J]. IEEE Transactions on Knowledge and Data Engineering,2020,32(11):2171-2184.
[25] SHEN Hengtao, LIU Luchen, YANG Yang, et al. Exploiting subspace relation in semantic labels for cross-modal hashing [J].IEEE Transactions Knowledge and Data Engineering,2021,33(10):3351-3365.
[26] MA Dekui, LIANG Jian, KONG Xiangwei, et al. Discrete cross-modal hashing for efficient multimedia retrieval[EB/OL].[2022-11-12].https://ieeexplore.ieee.org/document/7823584.
[27] CHEN Yong, ZHANG Hui, TIAN Zhibao, et al. Enhanced discrete multi-modal hashing: more constraints yet less time to learn [J]. IEEE Transactions on Knowledge and Data Engineering,2022,34(3):1177-1190.
[28] ZHANG Donglin, WU Xiaojun, LIU Zhen, et al. Fast discrete cross-modal hashing based on label relaxation and matrix factorization [EB/OL].[2022-11-20].https://ieeexplore.ieee.org/document/9412497.
[29] 滕少华,郭兰君,张巍,等.一种标签嵌入子空间的跨模态离散哈希学习 [J].江西师范大学学报(自然科学版),2021,45(3):305-313.
[30] 李鑫勇,滕少华,张巍,等.语义相似性保持的判别式跨模态哈 [J].计算机应用研究,2021,38(11):3359-3365.
[31] 敖宇翔,滕少华,张巍,等.标签局部结构保持的离散哈希方法 [J].小型微型计算机系统,2022, 43(5):998-1005.
[32] 庄智钧,滕少华,张巍,等.标签松弛回归的跨模态哈希检 [J].小型微型计算机系统,2022,43(10):2096-2105.
[33] TENG Shaohua, NING Chengzhen, ZHANG Wei, et al. Fast asymmetric and discrete cross-modal hashing with semantic consistency [EB/OL]. [2023-02-01].https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9853632.
[34] LIU Wei, MU Cun, KUMAR S, et al. Discrete graph hashing [EB/OL].[2022-11-21].https://dl.acm.org/doi/10.5555/2969033.2969208.
[35] BRYAn C,ANTONIO T,KEVIN P, et al. Labelme: a database and web-based tool for image annotation [J].International Journal of Computer Vision,2008,77(1/3):157-173.
[36] MARK J, MICHAEL S. The MIRFlickr retrieval evaluation [EB/OL].[2022-11-18].https://doi.org/10.1145/1460096.1460104.
[37] TATSENG C, TANG Jinhui, HONG Richang, et al. NUS-WIDE: a real-world Web image database from National University of Singapore [EB/OL].[2022-10-13].https://dl.acm.org/doi/10.1145/1646396.1646452.

备注/Memo

备注/Memo:
收稿日期:2022-12-09
基金项目:国家自然科学基金(61972102)资助项目.
作者简介:滕少华(1962—),男,江西南昌人,教授,博士,博士生导师,主要从事大数据、数据挖掘、人工智能、模式识别、智能制造和网络安全方面的研究.E-mail:shteng@gdut.edu.cn
更新日期/Last Update: 2023-05-25