特征混合增强与多损失融合的显著性目标检测

doi:10.3778/j.issn.1673-9418.2104104

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (10): 2395-2404.DOI: 10.3778/j.issn.1673-9418.2104104

特征混合增强与多损失融合的显著性目标检测

李春标, 谢林柏⁺(), 彭力

物联网技术应用教育部工程研究中心（江南大学物联网工程学院）,江苏无锡 214122

收稿日期:2021-04-20 修回日期:2021-06-07 出版日期:2022-10-01 发布日期:2021-06-11
通讯作者: + E-mail: xie_linbo@jiangnan.edu.cn
作者简介:李春标（1997—）,男,山东聊城人,硕士研究生,主要研究方向为显著性目标检测、深度学习。
谢林柏（1973—）,男,湖南永州人,博士,教授,博士生导师,CAA会员,主要研究方向为过程建模与控制、智能检测、系统安全性。
彭力（1967—）,男,河北唐山人,博士,教授,博士生导师,CAAI会员,CCF会员,主要研究方向为视觉物联网、行为识别、深度学习。
基金资助:
国家重点研发计划(2018YFD0400902);国家自然科学基金(61873112)

Salient Object Detection with Feature Hybrid Enhancement and Multi-loss Fusion

LI Chunbiao, XIE Linbo⁺(), PENG Li

Engineering Research Center of Internet of Things Technology Applications (School of Internet of Things Engineering, Jiangnan University), Ministry of Education, Wuxi, Jiangsu 214122, China

Received:2021-04-20 Revised:2021-06-07 Online:2022-10-01 Published:2021-06-11
About author:LI Chunbiao, born in 1997, M.S. candidate. His research interests include salient object detection and deep learning.
XIE Linbo, born in 1973, Ph.D., professor, Ph.D. supervisor, member of CAA. His reearch interests include process modeling and control, intelligent detection and system safety.
PENG Li, born in 1967, Ph.D., professor, Ph.D. supervisor, member of CAAI and CCF. His research interests include visual Internet of things, action recognition and deep learning.
Supported by:
National Key Research and Development Program of China(2018YFD0400902);National Natural Science Foundation of China(61873112)

摘要/Abstract

摘要：

针对当前显著性目标检测算法存在的特征缺失和区域一致性差的问题,基于全卷积神经网络提出一种特征混合增强与多损失融合的显著性目标检测算法。该算法包含上下文感知预测模块（CAPM）和特征混合增强模块（FHEM）。首先利用上下文感知预测模块提取图像多尺度特征信息,并且在预测模块中嵌入空间感知模块（SAM）以进一步提取图像高层语义信息,然后利用特征混合增强模块对预测模块产生的全局特征信息和细节特征信息进行有效的整合,并利用特征聚合模块（FAM）对整合的信息进行特征增强。此外,提出了一种新的区域增强损失函数（RA）,并将此损失函数与已有的二进制交叉熵（BCE）损失函数、结构化相似度（SSIM）损失函数融合,以多损失融合的方式监督网络保持前景区域的完整性以及增强区域像素一致性。在五个包含多个显著性目标和复杂背景的图像数据集上对算法进行验证,实验结果表明,该算法有效地提高了复杂场景下显著性目标的检测精度,改善了显著图特征缺失与区域一致性差的问题。

关键词: 卷积神经网络, 显著性目标检测, 特征混合增强, 区域增强损失

Abstract:

To tackle the problem of missing features and poor regional consistency in existing salient object detec-tion algorithms, a salient object detection network which uses feature hybrid enhancement and multi-loss fusion based on fully convolutional neural network is proposed. The network includes a context-aware prediction module (CAPM) and a feature hybrid enhancement module (FHEM). First, the context-aware prediction module is used to extract the multi-scale feature information of the image, in which the spatial-aware module (SAM) is embedded to further extract the high-level semantic information of the image. Furthermore, the feature hybrid enhancement module is used to effectively integrate the global feature information and the detailed feature information generated by the prediction module, and the integrated feature is enhanced through embedded feature aggregation module (FAM). In addition, the multi-loss fusion method is used to supervise the network, which combines the binary cross-entropy (BCE) loss function, the structured similarity (SSIM) loss function and the proposed regional augmentation (RA) loss function. The network with the multi-loss fusion method can maintain the integrity of the foreground region and enhance the regional pixel consistency. The algorithm is verified on five image datasets with multiple salient objects and complex backgrounds. Experimental results demonstrate that the algorithm effectively improves the detection accuracy of saliency objects in complex scenes, and alleviates the problem of saliency map features missing and poor regional consistency.

Key words: convolutional neural network, salient object detection, feature hybrid enhancement, regional aug-mentation loss

中图分类号:

TP391.4

李春标, 谢林柏, 彭力. 特征混合增强与多损失融合的显著性目标检测[J]. 计算机科学与探索, 2022, 16(10): 2395-2404.

LI Chunbiao, XIE Linbo, PENG Li. Salient Object Detection with Feature Hybrid Enhancement and Multi-loss Fusion[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(10): 2395-2404.

图/表 11

图1 显著性目标检测网络框架

Fig.1 Framework of salient object detection network

图2 空间感知模块

Fig.2 Spatial-aware module

图3 高效通道注意力模块

Fig.3 Efficient channel attention module

表1 算法使用不同模块性能比较

Table 1 Performance comparison of different modules in algorithm

Configurations	$m a x F β$	MAE
Baseline U-Net	0.742	0.089
CAPM	0.784	0.068
CAPM+FHEM(CA)	0.773	0.072
CAPM+FHEM(ECA)	0.803	0.057

表1 算法使用不同模块性能比较

Table 1 Performance comparison of different modules in algorithm

Configurations	$m a x F β$	MAE
Baseline U-Net	0.742	0.089
CAPM	0.784	0.068
CAPM+FHEM(CA)	0.773	0.072
CAPM+FHEM(ECA)	0.803	0.057

图4 不同模块显著性检测结果比较

Fig.4 Comparison of saliency detection results of different modules

表2 算法使用不同损失的性能比较

Table 2 Performance comparison of different losses in algorithm

Configurations	$m a x F β$	MAE
CAPM+FHEM+ $l b c e$	0.789	0.069
CAPM+FHEM+ $l b s$	0.787	0.064
CAPM+FHEM+ $l b s i$	0.792	0.066
CAPM+FHEM+ $l b s r$	0.803	0.057

表2 算法使用不同损失的性能比较

Table 2 Performance comparison of different losses in algorithm

Configurations	$m a x F β$	MAE
CAPM+FHEM+ $l b c e$	0.789	0.069
CAPM+FHEM+ $l b s$	0.787	0.064
CAPM+FHEM+ $l b s i$	0.792	0.066
CAPM+FHEM+ $l b s r$	0.803	0.057

图5 不同损失显著性检测结果比较

Fig.5 Comparison of saliency detection results of different losses

表3 算法使用不同系数RA损失的性能比较

Table 3 Performance comparison of RA loss with different coefficients in algorithm

$α$	$β$	$γ$	$m a x F β$	MAE
1.0	1.0	1.0	0.792	0.066
1.0	1.0	2.0	0.797	0.063
0.1	0.9	2.0	0.756	0.076
0.2	0.8	2.0	0.762	0.073
0.3	0.7	2.0	0.803	0.057
0.4	0.6	2.0	0.801	0.058
0.5	0.5	2.0	0.796	0.061

表3 算法使用不同系数RA损失的性能比较

Table 3 Performance comparison of RA loss with different coefficients in algorithm

$α$	$β$	$γ$	$m a x F β$	MAE
1.0	1.0	1.0	0.792	0.066
1.0	1.0	2.0	0.797	0.063
0.1	0.9	2.0	0.756	0.076
0.2	0.8	2.0	0.762	0.073
0.3	0.7	2.0	0.803	0.057
0.4	0.6	2.0	0.801	0.058
0.5	0.5	2.0	0.796	0.061

图6 不同算法的PR曲线与F-measure曲线比较

Fig.6 Comparison of PR curves and F-measure curves of different algorithms

表4 在5个数据集上不同算法性能比较

Table 4 Performance comparison of different algorithms on 5 datasets

Model	ECSSD		DUT-OMRON		DUTS-TE		HKU-IS		SOD
Model	$m a x F β$	MAE	$m a x F β$	MAE	$m a x F β$	MAE	$m a x F β$	MAE	$m a x F β$	MAE
Ours	0.942	0.036	0.803	0.057	0.860	0.046	0.928	0.031	0.857	0.102
CapsNet	0.887	0.052	0.703	0.063	0.799	0.048	0.880	0.039	—	—
RAS	0.921	0.056	0.786	0.062	0.831	0.059	0.913	0.045	0.850	0.124
Amulet	0.915	0.059	0.743	0.098	0.778	0.084	0.897	0.051	0.806	0.141
NLDF	0.905	0.063	0.753	0.080	0.812	0.066	0.902	0.048	0.841	0.124
UCF	0.911	0.078	0.734	0.132	0.771	0.117	0.886	0.074	0.803	0.164
RFCN	0.890	0.107	0.742	0.111	0.784	0.091	0.892	0.079	0.799	0.170
ELD	0.867	0.079	0.715	0.092	0.738	0.093	0.839	0.074	0.764	0.155
DCL	0.890	0.143	0.739	0.097	0.782	0.088	0.885	0.072	0.823	0.141
LEGS	0.827	0.118	0.669	0.133	0.655	0.138	0.766	0.119	0.734	0.196

表4 在5个数据集上不同算法性能比较

Table 4 Performance comparison of different algorithms on 5 datasets

Model	ECSSD		DUT-OMRON		DUTS-TE		HKU-IS		SOD
Model	$m a x F β$	MAE	$m a x F β$	MAE	$m a x F β$	MAE	$m a x F β$	MAE	$m a x F β$	MAE
Ours	0.942	0.036	0.803	0.057	0.860	0.046	0.928	0.031	0.857	0.102
CapsNet	0.887	0.052	0.703	0.063	0.799	0.048	0.880	0.039	—	—
RAS	0.921	0.056	0.786	0.062	0.831	0.059	0.913	0.045	0.850	0.124
Amulet	0.915	0.059	0.743	0.098	0.778	0.084	0.897	0.051	0.806	0.141
NLDF	0.905	0.063	0.753	0.080	0.812	0.066	0.902	0.048	0.841	0.124
UCF	0.911	0.078	0.734	0.132	0.771	0.117	0.886	0.074	0.803	0.164
RFCN	0.890	0.107	0.742	0.111	0.784	0.091	0.892	0.079	0.799	0.170
ELD	0.867	0.079	0.715	0.092	0.738	0.093	0.839	0.074	0.764	0.155
DCL	0.890	0.143	0.739	0.097	0.782	0.088	0.885	0.072	0.823	0.141
LEGS	0.827	0.118	0.669	0.133	0.655	0.138	0.766	0.119	0.734	0.196

图7 不同算法的显著性目标检测结果对比

Fig.7 Comparison of saliency detection results of different algorithms

参考文献 29

[1]	凌艳, 陈莹. 多尺度上下文信息增强的显著目标检测全卷积网络[J]. 计算机辅助设计与图形学学报, 2019, 31(11): 2007-2016.
	LING Y, CHEN Y. Salient object detection with multiscale context enhanced fully convolutional network[J]. Journal of Computer-Aided Design & Computer Graphics, 2019, 31(11): 2007-2016.
[2]	张守东, 杨明, 胡太. 基于多特征融合的显著性目标检测算法[J]. 计算机科学与探索, 2019, 13(5): 834-845. DOI
	ZHANG S D, YANG M, HU T. Salient object detection algorithm based on multi-feature fusion[J]. Journal of Fron-tiers of Computer Science and Technology, 2019, 13(5): 834-845.
[3]	OH S J, BENENSON R, KHOREVA A, et al. Exploiting sa-liency for object segmentation from image level labels[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 22-25, 2017. Was-hington: IEEE Computer Society, 2017: 5038-5047.
[4]	GAO Y, WANG M, TAO D C, et al. 3-D object retrieval and recognition with hypergraph analysis[J]. IEEE Transactions on Image Processing, 2012, 21(9): 4290-4303. DOI PMID
[5]	REN Z X, GAO S H, CHIA L T, et al. Region-based saliency detection and its application in object recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(5): 769-779. DOI URL
[6]	ZHANG F, DU B, ZHANG L P. Saliency-guided unsupervised feature learning for scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(4): 2175-2184. DOI URL
[7]	于明, 李博昭, 于洋, 等. 基于多图流形排序的图像显著性检测[J]. 自动化学报, 2019, 45(3): 577-592.
	YU M, LI B Z, YU Y, et al. Image saliency detection with multi-graph model and manifold ranking[J]. Acta Automatica Sinica, 2019, 45(3): 577-592.
[8]	常振, 段先华, 鲁文超, 等. 基于多尺度的贝叶斯模型显著性检测[J]. 计算机工程与应用, 2020, 56(11): 207-213. DOI
	CHANG Z, DUAN X H, LU W C, et al. Multi-scale saliency detection based on Bayesian framework[J]. Computer Eng-ineering and Applications, 2020, 56(11): 207-213.
[9]	时斐斐, 张松龙, 彭力. 结合边缘特征先验引导的深度卷积显著性检测[J]. 计算机工程与应用, 2020, 56(14): 199-206. DOI
	SHI F F, ZHANG S L, PENG L. Deep convolution saliency detection combined with edge feature prior-based inspec-tion[J]. Computer Engineering and Applications, 2020, 56(14): 199-206.
[10]	SCHARFENBERGER C, WONG A, FERGANI K, et al. Statistical textural distinctiveness for salient region detection in natural images[C]// Proceedings of the 2013 IEEE Con-ference on Computer Vision and Pattern Recognition, Port-land, Jun 23-28, 2013. Washington: IEEE Computer Society, 2013: 979-986.
[11]	KIM J H, HAN D Y, TAI Y W, et al. Salient region detec-tion via high-dimensional color transform[C]// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Jun 24-27, 2014. Washington: IEEE Computer Society, 2014: 883-890.
[12]	ZHU W J, LIANG S, WEI Y C, et al. Saliency optimization from robust background detection[C]// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Re-cognition, Columbus, Jun 24-27, 2014. Washington: IEEE Computer Society, 2014: 2814-2821.
[13]	LIU Y, ZHANG Q, ZHANG D W, et al. Employing deep part-object relationships for salient object detection[C]// Pro-ceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1232-1241.
[14]	KUEN J, WANG Z H, WANG G. Recurrent attentional net-works for saliency detection[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recogni-tion, Las Vegas, Jun 26-Jul 1, 2016. Washington: IEEE Computer Society, 2016: 3668-3677.
[15]	ZHANG P P, WANG D, LU H C, et al. Amulet: aggregating multi-level convolutional features for salient object detec-tion[C]// Proceedings of the 2017 IEEE International Con-ference on Computer Vision, Venice, Oct 22-29, 2017. Wa-shington: IEEE Computer Society, 2017: 202-211.
[16]	LUO Z M, MISHRA A K, ACHKAR A, et al. Non-local deep features for salient object detection[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 6593-6601.
[17]	ZHANG P P, WANG D, LU H C, et al. Learning uncertain convolutional features for accurate saliency detection[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 212-221.
[18]	WANG L Z, WANG L J, LU H C, et al. Salient object detec-tion with recurrent fully convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(7): 1734-1746. DOI URL
[19]	WANG Z, SIMONCELLI E P, BOVIK A C. Multiscale struc-tural similarity for image quality assessment[C]// Proceedings of the 2003 IEEE Asilomar Conference on Signals, Pacific Grove, Nov 9-12, 2003. Washington: IEEE Computer Society, 2003.
[20]	FENG M Y, LU H C, DING E. Attentive feedback network for boundary-aware salient object detection[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1623-1632.
[21]	QIN X B, ZHANG Z C, HUANG C Y, et al. BASNet: boun-dary-aware salient object detection[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Re-cognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 7471-7481.
[22]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convo-lutional networks for biomedical image segmentation[C]// Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Mu-nich, Oct 5-9, 2015. Cham: Springer, 2015: 234-241.
[23]	ZHAO T, WU X Q. Pyramid feature attention network for saliency detection[C]// Proceedings of the 2019 IEEE Confe-rence on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 3080-3089.
[24]	ZHANG L H, WU J, WANG T T, et al. A multistage refine-ment network for salient object detection[J]. IEEE Transac-tions on Image Processing, 2020, 29: 3534-3545.
[25]	WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 11531-11539.
[26]	ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency-tuned salient region detection[C]// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recogni-tion, Florida, Jun 20-25, 2009. Washington: IEEE Computer Society, 2009: 1597-1604.
[27]	LEE G, TAI Y W, KIM J. Deep saliency with encoded low level distance map and high level features[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pat-tern Recognition, Las Vegas, Jun 26-Jul 1, 2016. Washing-ton: IEEE Computer Society, 2016: 660-668.
[28]	LI G B, YU Y Z. Deep contrast learning for salient object detection[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 26-Jul 1, 2016. Washington: IEEE Computer Society, 2016: 478-487.
[29]	WANG L J, LU H C, RUAN X, et al. Deep networks for sa-liency detection via local estimation and global search[C]// Proceedings of the 2015 IEEE Conference on Computer Vi-sion and Pattern Recognition, Boston, Jun 7-12, 2015. Wa-shington: IEEE Computer Society, 2015: 3183-3192.

特征混合增强与多损失融合的显著性目标检测

Salient Object Detection with Feature Hybrid Enhancement and Multi-loss Fusion

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 29

相关文章 15

编辑推荐

Metrics

[1]	安凤平, 李晓薇, 曹翔. 权重初始化-滑动窗口CNN的医学图像分类[J]. 计算机科学与探索, 2022, 16(8): 1885-1897.
[2]	欧阳柳, 贺禧, 瞿绍军. 全卷积注意力机制神经网络的图像语义分割[J]. 计算机科学与探索, 2022, 16(5): 1136-1145.
[3]	程卫月, 张雪琴, 林克正, 李骜. 融合全局与局部特征的深度卷积神经网络算法[J]. 计算机科学与探索, 2022, 16(5): 1146-1154.
[4]	童敢, 黄立波. Winograd快速卷积相关研究综述[J]. 计算机科学与探索, 2022, 16(5): 959-971.
[5]	裴利沈, 赵雪专. 群体行为识别深度学习方法研究综述[J]. 计算机科学与探索, 2022, 16(4): 775-790.
[6]	陆仲达, 张春达, 张佳奇, 王子菲, 许军华. 双分支网络的苹果叶部病害识别[J]. 计算机科学与探索, 2022, 16(4): 917-926.
[7]	肖泽管, 陈清亮. 融合多种类型语法信息的属性级情感分析模型[J]. 计算机科学与探索, 2022, 16(2): 395-402.
[8]	李朝阳, 李琳, 陶晓辉. 面向动态交通流预测的双流图卷积网络[J]. 计算机科学与探索, 2022, 16(2): 384-394.
[9]	李志欣, 陈圣嘉, 周韬, 马慧芳. 协同级联网络和对抗网络的目标检测[J]. 计算机科学与探索, 2022, 16(1): 217-230.
[10]	任龙杰, 孙颖, 丁卫平, 鞠恒荣, 曹金鑫. 基于单种群蛙跳优化CNN的眼底图像多病变检测[J]. 计算机科学与探索, 2021, 15(9): 1762-1772.
[11]	张梦倩, 张莉. 粗-细两阶段卷积神经网络算法[J]. 计算机科学与探索, 2021, 15(8): 1501-1510.
[12]	方钧婷, 谭晓阳. 注意力级联网络的金属表面缺陷检测算法[J]. 计算机科学与探索, 2021, 15(7): 1245-1254.
[13]	能文鹏, 陆军, 赵彩虹. 基于关系归纳偏置的睡眠分期综述[J]. 计算机科学与探索, 2021, 15(6): 1026-1037.
[14]	赵小强, 徐慧萍. 分级特征融合的图像语义分割[J]. 计算机科学与探索, 2021, 15(5): 949-957.
[15]	马丹, 万良, 程琪芩, 孙志强. Attention-CNN在恶意代码检测中的应用研究[J]. 计算机科学与探索, 2021, 15(4): 670-681.