Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (8): 1865-1876.DOI: 10.3778/j.issn.1673-9418.2012041

• Graphics and Image • Previous Articles     Next Articles

Salient Instance Segmentation via Multiscale Boundary Characteristic Network

HE Li, ZHANG Hongyan(), FANG Wanlin   

  1. College of Science and Technology, Tianjin University of Finance and Economics, Tianjin 300222, China
  • Received:2020-12-11 Revised:2021-02-04 Online:2022-08-01 Published:2021-03-03
  • About author:HE Li, born in 1969, Ph.D., professor, M.S. supervisor, professional member of CCF. Her research interests include data mining, machine learning, etc.
    ZHANG Hongyan, born in 1996, M.S. candidate. Her research interests include computer vision, image segmentation, etc.
    FANG Wanlin, born in 1997, M.S. candidate. Her research interests include neural network, natural language processing, sentiment analysis, etc.
  • Supported by:
    the National Natural Science Foundation of China(11701410);the Natural Science Foundation of Tianjin(18JCYBJC85100)

融合多尺度边界特征的显著实例分割

何丽, 张红艳(), 房婉琳   

  1. 天津财经大学 理工学院,天津 300222
  • 通讯作者: +E-mail: zhy16622553596@163.com
  • 作者简介:何丽(1969—),女,安徽舒城人,博士,教授,硕士生导师,CCF专业会员,主要研究方向为数据挖掘、机器学习等。
    张红艳(1996—),女,山西阳泉人,硕士研究生,主要研究方向为计算机视觉、图像分割等。
    房婉琳(1997—),女,黑龙江哈尔滨人,硕士研究生,主要研究方向为神经网络、自然语言处理、情感分析等。
  • 基金资助:
    国家自然科学基金(11701410);天津市自然科学基金(18JCYBJC85100)

Abstract:

Locating objects of interest is a basic task in the application of computer vision. Salient instance segmentation can obtain instance of interest by detecting visually significant objects and segmenting them at pixel level. In order to utilize the ability of feature separation between target object and its surrounding background, single stage salient-instance segmentation (S4Net) designs a new region feature extraction layer called ROIMasking. For the characteristics of convolutional neural network, repeated convolution and upsampling will result in the loss of boundary information, rough boundary segmentation and the reduction of segmentation accuracy. To solve this problem, using the target edge detection method, a new end-to-end salient instance segmentation via multiscale boun-dary characteristic network (MBCNet) based on S4Net is proposed. This method designs a multiscale boundary feature extraction branch. A boundary refinement block with hybrid dilation convolution and residual network structure is used to enhance the extraction of the instance boundary information. The MBCNet sharing layers realize to transfer the boundary information. At the same time, in order to promote the accuracy of segmentation, a new boundary-segmentation joint loss function is proposed, realizing synchronous training of target boundary feature ext-raction and instance segmentation in the same network. Experimental results show that, compared with S4Net, the mAP0.5 and mAP0.7 of the proposal are 88.90% and 67.94% on the saliency instance dataset, with the improvement of 2.20 percentage points and 4.24 percentage points, respectively.

Key words: salient instance segmentation, edge detection, boundary refinement, boundary-segmentation joint training

摘要:

对感兴趣的对象进行定位是计算机视觉应用的一个基础任务。显著实例分割通过对视觉上具有显著性的物体进行检测并对其进行像素级分割,可以获得感兴趣的实例类。单阶段显著实例分割网络(S4Net)为利用目标对象和其周围背景的特征分离能力,设计了一个新的区域特征抽取层ROIMasking。但由于卷积神经网络自身的特性,多次的卷积和上采样会造成实例边界信息缺失,导致边界分割粗糙,影响分割的精度。为了解决显著实例分割中的边界信息丢失问题,在S4Net的基础上借鉴目标边缘检测方法,提出了一种结合边界特征的端到端显著实例分割方法(MBCNet)。该方法设计了一个多尺度融合的边界特征提取分支,利用带有混合空洞卷积和残差网络结构的边界细化模块强化对实例边界信息的提取,并通过网络共享层实现了边界信息的传递;同时,为提高分割的精度,提出了一个新的边界-分割联合损失函数,实现了在同一个网络中对目标边界特征提取分支和实例分割分支的同步训练。实验结果显示,提出的方法在saliency instance数据集上的mAP0.5和mAP0.7分别达到88.90%和67.94%,比目前主流的显著实例分割方法S4Net分别提升了2.20个百分点和4.24个百分点。

关键词: 显著实例分割, 边缘检测, 边界细化, 边界-分割联合训练

CLC Number: