Multi-scale and Boundary Fusion Network for Skin Lesion Regions Segmentation

doi:10.3778/j.issn.1673-9418.2306003

Abstract

Abstract: Accurate segmentation of skin lesion regions is a key step in clinical diagnosis and analysis. Aiming at the poor segmentation effect of the existing networks in skin lesion regions due to the presence of variable size, irregular shape, fuzzy boundaries and obscured lesion regions, a multi-scale and boundary fusion network (MSBF-Net) for skin lesion region segmentation is proposed by improving the original structure based on the U-Net. Specifically, firstly, a split pooling (SplitPool) module is proposed, which effectively solves the problem of spatial information loss while reducing the image resolution. Secondly, a full-scale feature fusion (FSFF) module is proposed, which effectively solves the problem that the previous methods only fuse the deep features to the shallow features, while ignoring the contribution of the detail information in the more shallow features to the network segmentation decision. Meanwhile, the original jump connections of U-Net are redesigned to provide richer contextual information for the decoder. Finally, sub-paths for enhancing the network’s ability to learn boundary features are proposed, and the boundary fusion (BF) module is introduced to fuse the prediction results of the main paths and sub-paths, which effectively solves the problems of irregular shape and boundary ambiguity of the lesion region. Dice and JI reach 90.12% and 83.61% on the ISIC2018 dataset, which are 1.13 percentage points and 1.62 percentage points higher than the baseline network, respectively; Dice and JI reach 94.72% and 90.18% on the PH2 dataset, which are 1.49 percentage points and 2.17 percentage points higher than the baseline network, respectively. Experimental results show that MSBF-Net significantly improves the accuracy of skin lesion region segmentation and exceeds the existing state-of-the-art methods in several indices, further validating the effectiveness of the method.

Key words: skin lesion region segmentation, skip connections, boundary feature, feature fusion, attention mechanism

摘要： 皮肤病变区域的准确分割是临床诊断分析的关键一步。针对现有网络在皮肤病变区域存在尺寸大小多变、形状不规则、边界模糊和病变区域被遮挡的情况导致的分割效果不佳问题，在U-Net的基础上改进了原有结构，提出了一种用于皮肤病变区域分割的多尺度和边界融合网络（MSBF-Net）。首先，提出了分裂池化（SplitPool）模块，在缩小图像分辨率的同时有效地解决了空间信息丢失的问题。其次，提出了全尺度特征融合（FSFF）模块，有效地解决了以往方法仅将深层特征向浅层特征融合，而忽略了更浅层特征中的细节信息对网络分割决策的贡献问题。同时，重新设计了U-Net原有的跳跃连接，为解码器提供了更丰富的上下文信息。最后，提出了用于增强网络对边界特征学习能力的子路径，并引入边界融合（BF）模块将主路径和子路径的预测结果进行融合，有效地解决了病变区域形状不规则和边界模糊问题。在ISIC2018数据集上，Dice和JI分别达到了90.12%和83.61%，比基线网络分别提高了1.13个百分点和1.62个百分点；在PH2数据集上，Dice和JI分别达到了94.72%和90.18%，比基线网络分别提高了1.49个百分点和2.17个百分点。实验结果表明，MSBF-Net显著提升了皮肤病变区域分割的精确度，并在多个指标上超过了现有的先进方法，进一步验证了方法的有效性。

关键词: 皮肤病变区域分割, 跳跃连接, 边界特征, 特征融合, 注意力机制

WANG Guokai, ZHANG Xiang, WANG Shunfang. Multi-scale and Boundary Fusion Network for Skin Lesion Regions Segmentation[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(7): 1826-1837.

王国凯, 张翔, 王顺芳. 多尺度和边界融合的皮肤病变区域分割网络[J]. 计算机科学与探索, 2024, 18(7): 1826-1837.

References

[1] FERLAY J, COLOMBET M, SOERJOMATARAM I, et al. Cancer statistics for the year 2020: an overview[J]. International Journal of Cancer, 2021, 149(4): 778-789.
[2] ROGERS H W, WEINSTOCK M A, FELDMAN S R, et al. Incidence estimate of nonmelanoma skin cancer (keratinocyte carcinomas) in the US population, 2012[J]. JAMA Dermatology, 2015, 151(10): 1081-1086.
[3] KHARAZMI P, ALJASSER M I, LUI H, et al. Automated detection and segmentation of vascular structures of skin lesions seen in dermoscopy, with an application to basal cell carcinoma classification[J]. IEEE Journal of Biomedical and Health Informatics, 2016, 21(6): 1675-1684.
[4] JAIN A, WAY D, GUPTA V, et al. Development and assessment of an artificial intelligence-based tool for skin condition diagnosis by primary care physicians and nurse practitioners in teledermatology practices[J]. JAMA Network Open, 2021, 4(4): e217249.
[5] YUEKSEL M E, BORLU M. Accurate segmentation of dermoscopic images by image thresholding based on type-2 fuzzy logic[J]. IEEE Transactions on Fuzzy Systems, 2009, 17(4): 976-982.
[6] MA Z, TAVARES J M R S. A novel approach to segment skin lesions in dermoscopic images based on a deformable model[J]. IEEE Journal of Biomedical and Health Informatics, 2015, 20(2): 615-623.
[7] MIRBEIK A, ASHINOFF R, JONG T, et al. Real-time high-resolution millimeter-wave imaging for in-vivo skin cancer diagnosis[J]. Scientific Reports, 2022, 12(1): 4971.
[8] LI H, CHEN D, NAILON W H, et al. Dual convolutional neural networks for breast mass segmentation and diagnosis in mammography[J]. IEEE Transactions on Medical Imaging, 2021, 41(1): 3-13.
[9] DI S, ZHAO Y Q, LIAO M, et al. TD-Net: a hybrid end-to-end network for automatic liver tumor segmentation from CT images[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 27(3): 1163-1172.
[10] 梁礼明, 陈鑫, 余洁, 等. 多尺度注意力细化视网膜分割算法[J]. 计算机工程与应用，2023, 59(6): 212-220.
LIANG L M, CHEN X, YU J, et al. Multi-scale attention refinement retinal segmentation algorithm[J]. Computer Engineering and Applications, 2023, 59(6): 212-220.
[11] DAI D, DONG C, XU S, et al. Ms RED: a novel multi-scale residual encoding and decoding network for skin lesion segmentation[J]. Medical Image Analysis, 2022, 75: 102293.
[12] HUANG X, DENG Z, LI D, et al. MISSFormer: an effective transformer for 2D medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2023, 42(5): 1484-1494.
[13] QU Z, ZHUO L, CAO J, et al. TP-Net: two-path network for retinal vessel segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2023, 27(4): 1979-1990.
[14] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Oct 5-9, 2015. Cham: Springer, 2015: 234-241.
[15] GU R, WANG G, SONG T, et al. CA-Net: comprehensive attention convolutional neural networks for explainable medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2021, 40(2): 699-711.
[16] HU K, LU J, LEE D, et al. AS-Net: attention synergy network for skin lesion segmentation[J]. Expert Systems with Applications, 2022, 201: 117112.
[17] WU H, CHEN S, CHEN G, et al. FAT-Net: feature adaptive transformers for automated skin lesion segmentation[J]. Medical Image Analysis, 2022, 76: 102327.
[18] BASAK H, KUNDU R, SARKAR R. MFSNet: a multi focus segmentation network for skin lesion segmentation[J]. Pattern Recognition, 2022, 128: 108673.
[19] CAO W, YUAN G, LIU Q, et al. ICL-Net: global and local inter-pixel correlations learning network for skin lesion segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2023, 27(1): 145-156.
[20] FENG K, REN L, WANG G, et al. SLT-Net: a codec network for skin lesion segmentation[J]. Computers in Biology and Medicine, 2022, 148: 105942.
[21] SUN Y, DAI D, ZHANG Q, et al. MSCA-Net: multi-scale contextual attention network for skin lesion segmentation[J]. Pattern Recognition, 2023, 139: 109524.
[22] ZHOU T, ZHOU Y, HE K, et al. Cross-level feature aggregation network for polyp segmentation[J]. Pattern Recognition, 2023, 140: 109555.
[23] FENG S, ZHAO H, SHI F, et al. CPFNet: context pyramid fusion network for medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(10): 3008-3018.
[24] CODELLA N, ROTEMBERG V, TSCHANDL P, et al. Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC)[EB/OL]. [2023-03-20]. https://arxiv.org/abs/1902.03368.
[25] TSCHANDL P, ROSENDAHL C, KITTLER H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions[J]. Scientific Data, 2018, 5(1): 1-9.
[26] MENDON?A T, FERREIRA P M, MARQUES J S, et al. PH 2-A dermoscopic image database for research and bench-marking[C]//Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Piscataway: IEEE, 2013: 5437-5440.
[27] ZHOU Z, SIDDIQUEE R M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[C]//Proceedings of the 4th International Workshop on Deep Learning in Medical Image Analysis and the 8th International Workshop on Multimodal Learning for Clinical Decision Support, Held in Conjunction with 2018 International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Sep 20, 2018. Cham: Springer, 2018: 3-11.
[28] OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: learning where to look for the pancreas[EB/OL].[2023-03-20]. https://arxiv.org/abs/1804.03999.
[29] GU Z, CHENG J, FU H, et al. CE-Net: context encoder network for 2D medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2019, 38(10): 2281-2292.