基于MSFA-Net的肝脏CT图像分割方法

doi:10.3778/j.issn.1673-9418.2106020

摘要/Abstract

摘要： 从患者的腹部CT图像中自动分割出肝脏对于肝脏疾病的诊断意义重大。由于在U-Net中使用自下而上的特征融合方式忽略了低级特征的重要性，导致网络分割性能较差，以及肝脏与相邻器官组织的灰度值较为相似，使得一些微小的细节特征不易被关注。针对以上问题，提出了一种基于多尺度语义特征融合和注意力机制的肝脏分割网络（MSFA-Net）。首先，使用空洞残差卷积（DRC）捕获多尺度特征；然后，采用MSFA模块将自上而下和自下而上的多尺度特征融合方法与注意力机制相结合，来充分融合多尺度特征和关注微小特征；最后，通过深度监督（DS）对特征图求和来提升分割效果。在MICCAI2017 LiTS和3DIRCADb数据集上进行了消融研究，在LiTS数据集上获得了0.961和0.965的DC和DG评分，比基线网络分别提高了3.4%和2.0%；在3DIRCADb数据集上，其DC和DG评分同为0.965，比基线网络分别提高了3.5%和3.3%。

关键词: 肝脏分割, U-Net, 多尺度特征融合, 注意力机制, 深度监督

Abstract: Automatic segmentation of the liver from the patient’s abdominal CT images is significant for the diagnosis of liver disease. Since the bottom-up feature fusion methods in U-Net ignore the importance of low-level features, the segmentation results are not satisfactory. In addition, the gray values for the liver and its adjacent organs are very similar, it is therefore difficult to distinguish some tiny detail features. To address the above problems, a liver segmentation network based on multi-scale semantic features fusion and attention mechanism (MSFA-Net) is proposed. Firstly, dilated residual convolution (DRC) is used to capture multi-scale features. Then, the MSFA module combines top-down and bottom-up multi-scale feature fusion methods with the attention mecha-nism to fully fuse multi-scale features and pay attention to tiny features. Finally, the feature maps are summed by deep supervise (DS) to improve the segmentation effect. The ablation study is performed on the MICCAI2017 LiTS and 3DIRCADb datasets. The dice per case (DC) and dice global (DG) scores of 0.961 and 0.965 are obtained on the LiTS dataset, with an improvement of 3.4% and 2.0% respectively compared with baseline network. Both DC and DG scores of 0.965 are obtained on the 3DIRCADb dataset, with an increase of 3.5% and 3.3% respectively compared with baseline network.

Key words: liver segmentation, U-Net, multi-scale feature fusion, attention mechanism, deep supervision

沈怀艳, 吴云. 基于MSFA-Net的肝脏CT图像分割方法[J]. 计算机科学与探索, 2023, 17(3): 646-656.

SHEN Huaiyan, WU Yun. Liver CT Image Segmentation Method Based on MSFA-Net[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(3): 646-656.

参考文献

[1] TAGHANNKI S A，ABHISHEK K, COHEN J P, et al. Deep semantic segmentation of natural and medical images: a review[J]. Artificial Intelligence Review, 2021(1): 1-42.
[2] YAN J, SCHWARTZ L H, ZHAO B. Semiautomatic segmen-tation of liver metastases on volumetric CT images[J]. Medical Physics, 2015, 42(11): 6283-6293.
[3] LONG J, SHELHAMER E, DARRELL T. Fully convol-utional networks for semantic segmentation[J]. IEEE Trans-actions on Pattern Analysis and Machine Intelligence, 2015, 39(4): 640-651.
[4] RONNEBERGER O, FISCHER P, BROX T, et al. U-NET: convolutional networks for biomedical image segmentation[C]//LNCS 9351: Proceedings of the 2015 International Conference on Medical Image Computing and Computer-assisted Intervention, Munich, Oct 5-9, 2015. Cham: Springer, 2015: 234-241.
[5] LIU Z, SONG Y Q, SHENG V S, et al. Liver CT sequence segmentation based with improved U-Net and graph cut[J]. Expert Systems with Applications, 2019, 126: 54-63.
[6] 张泽林, 李宝明, 徐军. 基于条件生成对抗网络的三维肝脏及肿瘤区域自动分割[J]. 生物医学工程学杂志, 2021, 38(1): 80-88.
ZHANG Z L, LI B M, XU J. Automatic three-dimensional segmentation of liver and tumors regions based on cond-itional generative adversarial networks[J]. Journal of Bio-medical Engineering, 2021, 38(1): 80-88.
[7] WANG J, LV P, WANG H, et al. SAR-U-Net: squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-Net for automatic liver CT segmentation[J]. arXiv: 2103.06419, 2021.
[8] LU F, WU F, HU P, et al. Automatic 3D liver location and segmentation via convolutional neural network and graph cut[J]. International Journal of Computer Assisted Radiology and Surgery, 2017, 12(2): 171-182.
[9] LEI T, ZHOU W Z, ZHANG Y X, et al. Lightweight V-Net for liver segmentation[C]//Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, May 4-8, 2020. Piscataway: IEEE, 2020: 1379-1383.
[10] ZHANG C Y, AI D, FENG C, et al. Dial/hybrid cascade 3DResUNet for liver and tumor segmentation[C]//Proce-edings of the 4th International Conference on Digital Signal Processing, Chengdu, Jun 19-21, 2020. New York: ACM, 2020: 92-96.
[11] LIU T, LIU J, MA Y, et al. Spatial feature fusion convo-lutional network for liver and liver tumor segmentation from CT images[J]. Medical Physics, 2020, 48(1): 264-272.
[12] CHEN X Y, ZHANG R, YAN P K. Feature fusion encoder decoder network for automatic liver lesion segmentation[C]//Proceedings of the 16th IEEE International Sympo-sium on Biomedical Imaging, Venice, Apr 8-11, 2019. Pis-cataway: IEEE, 2019: 430-433.
[13] FENG S, ZHAO H, SHI F, et al. CPFNet: context pyramid fusion network for medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(10): 3008-3018.
[14] XU K, BA J, KIROS R, et al. Show, attend and tell: neural image caption generation with visual attention[C]//Proce-edings of the 32nd International Conference on Intern-ational Conference on Machine Learning, Lille France, Jul 6-11, 2015: 2048-2057.
[15] 刘云鹏, 刘光品, 王仁芳, 等. 深度学习结合影像组学的肝脏肿瘤CT分割[J]. 中国图象图形学报, 2020, 25(10): 2128-2141.
LIU Y P, LIU G P, WANG R F, et al. Accurate segmentation method of liver tumor CT based on the combination of deep learning and radiomics[J]. Journal of Image and Grap-hics, 2020, 25(10): 2128-2141.
[16] SCHLEMPER J, OKTAY O, SCHAAP M, et al. Attention gated networks: learning to leverage salient regions in medical images[J]. Medical Image Analysis, 2019, 53: 197-207.
[17] JIN Q G, MENG Z P, SUN C M, et al. RA-UNet: a hybrid deep attention-aware network to extract liver and tumor in CT scans[J]. Frontiers in Bioengineering and Biotechnology, 2020, 8: 1471.
[18] JIANG H, SHI T, BAI Z, et al. AhcNet: an application of attention mechanism and hybrid connection for liver tumor segmentation in CT volumes[J]. IEEE Access, 2019, 7: 24898- 24909.
[19] XU Y, CAI M, LIN L, et al. PA-ResSeg: a phase attention residual network for liver tumor segmentation from multi-phase CT images[J]. Medical Physics, 2021, 48(7):?3752-3766.
[20] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2017, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 5998-6008.
[21] CHEN J, LU Y, YU Q, et al. TransUNet: transformers make strong encoders for medical image segmentation[J]. arXiv:2102.04306, 2021.
[22] YU F, KOLTUN V, FUNKHOUSER T A. Dilated residual networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 636-644.
[23] HU Y, LI J, HUANG Y, et al. Channel-wise and spatial feature modulation network for single image super-resolu-tion[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30(11): 3911-3927.
[24] LEE C Y, XIE S N, GALLAGHER P, et al. Deeply-super-vised nets[C]//Proceedings of the 18th International Confe-rence on Artificial Intelligence and Statistics, San Diego, May 9-12, 2015: 562-570.
[25] BILIC P, CHRIST P F, VORONTSOV E, et al. The liver tumor segmentation benchmark (LiTS)[J]. arXiv:1901.04056, 2019.
[26] SOLER L, HOSTETTLER A, AGNUS V, et al. 3D image reconstruction for comparison of algorithm database: a patient specific anatomical and medical image database[R]. Strasbourg: IRCAD, 2010.
[27] TAHA A A, HANBURY A. Metrics for evaluating 3D medi-cal image segmentation: analysis, selection, and tool[J]. BMC Medical Imaging, 2015, 15(1): 1-28.
[28] 徐宝泉, 凌彤辉. 基于级联Vnet-S网络的CT影像单一器官自动分割算法[J]. 计算机应用, 2019, 39(8): 2420-2425.
XU B Q, LING T H. Automatic segmentation algorithm for single organ of CT images based on cascaded Vnet-S network[J]. Journal of Computer Applications, 2019, 39(8): 2420-2425.
[29] 黄泳嘉, 史再峰, 王仲琦, 等. 基于混合损失函数的改进型U-Net肝部医学影像分割方法[J]. 激光与光电子学进展, 2020, 57(22): 66-75.
HUANG Y J, SHI Z F, WANG Z Q, et al. Improved U-Net based on mixed loss function for liver medical image seg-mentation[J]. Laser & Optoelectronics Progress, 2020, 57(22): 66-75.
[30] SEO H, HUANG C, BASSENNE M, et al. Modified U-Net (mU-Net) with incorporation of object-dependent high level features for improved liver and liver-tumor segmentation in CT images[J]. IEEE Transactions on Medical Imaging, 2019, 39(5): 1316-1325.
[31] ZHANG Y, PAN X, LI C, et al. 3D liver and tumor seg-mentation with CNNs based on region and distance metrics[J]. Applied Sciences, 2020, 10(11): 3794.
[32] BUDAK ü, GUO Y, TANYILDIZI E, et al. Cascaded deep convolutional encoder-decoder neural networks for efficient liver tumor segmentation[J]. Medical Hypotheses, 2020, 134: 109431.
[33] XIE X, ZHANG W, WANG H, et al. Dynamic adaptive residual network for liver CT image segmentation[J]. Com-puters & Electrical Engineering, 2021, 91: 10702.
[34] HAN Y, LI X, WANG B, et al. Boundary loss-based 2.5D fully convolutional neural networks approach for segmen-tation: a case study of the liver and tumor on computed tomography[J]. Algorithms, 2021, 14(5): 144.