Multi-model Fusion of Coronary CTA Segmentation Based on Attention Mechanism

doi:10.3778/j.issn.1673-9418.1909028

Abstract

Abstract:

Coronary heart disease is one of the biggest health problems in the world, so the early prevention and diagnosis of coronary heart disease is very important. The presence of plaque and coronary artery stenosis are the main causes of coronary heart disease. The detection of plaque and coronary artery segmentation have become the first choice for detecting coronary artery disease. At present, manual coronary artery segmentation is time-consuming and determined by the operator??s subjective consciousness, which makes the need for automatic segmentation technology obvious in clinical diagnosis. In this paper, a method of computed tomography angiography (CTA) based on deep learning multi-model fusion is proposed, which includes three network models: an original 3D fully convolutional network (3D FCN) and two networks embedded the attention gate (AG) model in the original 3D FCN. And then the prediction results of the three networks are fused by majority voting algorithm to obtain the final result of the network prediction. In the post-processing stage, the level set function is used to further iteratively optimize the network fusion prediction results. In the process of evaluating the proposed method, Jaccard index (JI) and Dice similarity coefficient (DSC) scores are compared as performance measures. The final results show that the proposed method provides better and more accurate segmentation results.

Key words: coronary computed tomography angiography (CTA), multi-model fusion, three-dimensional fully convolutional network (3D FCN), attention gate model, level set function

摘要：

冠心病是当今世界上最大的健康问题之一，因此对冠心病的早期预防和诊断非常重要。斑块的存在和冠状动脉血管狭窄是引发冠心病的主要原因，对斑块的检测和冠状动脉血管分割成为检测冠状动脉疾病的首选方案。目前手动分割冠状动脉耗时并且由操作者的主观意识决定，这使得现在的临床医学诊断中对自动分割技术的需要显而易见。提出了一种基于深度学习多模型融合的冠脉CT血管造影（CTA）的血管分割方法，该方法包括三个网络模型：一个原始三维全卷积网络（3D FCN），以及两个在原始3D FCN中嵌入注意力门控（AG）模型的网络。然后把三种网络的预测结果采用多数投票算法进行融合来得到网络预测的最终结果。在后处理阶段，采用水平集函数对网络融合预测的结果进行进一步迭代优化。在评估所提出方法的过程中把JI和DSC分数来作为性能度量进行比较，最终结果表明，所提出的方法提供了更好更准确的分割结果。

关键词: 冠脉CTA, 多模型融合, 三维全卷积网络（3D FCN）, 注意力门控模型, 水平集函数

SHEN Ye, FANG Zhijun, GAO Yongbin. Multi-model Fusion of Coronary CTA Segmentation Based on Attention Mechanism[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(9): 1602-1611.

沈烨，方志军，高永彬. 嵌入注意力机制的多模型融合冠脉CTA分割算法[J]. 计算机科学与探索, 2020, 14(9): 1602-1611.

References

[1] Nabel E G, Braunwald E. A tale of coronary artery disease and myocardial infarction[J]. New England Journal of Medi-cine, 2012, 366(1): 54-63.
[2] Shakeri M, Tsogkas S, Ferrante E, et al. Sub-cortical brain structure segmentation using F-CNN??s[C]//Proceedings of the 13th IEEE International Symposium on Biomedical Imaging, Prague, Apr 13-16, 2016. Piscataway: IEEE, 2016: 269-272.
[3] Soomro M H, Giunta G, Laghi A, et al. Segmenting MR ima-ges by level-set algorithms for perspective colorectal cancer diagnosis[C]//Proceedings of the 2017 VI ECCOMAS The-matic Conference on Computational Vision and Medical Image Processing Porto, Oct 18-20, 2017. Berlin, Heidelberg: Sprin-ger, 2017: 396-406.
[4] Chen Y T. A novel approach to segmentation and measure-ment of medical image using level set methods[J]. Magnetic Resonance Imaging, 2017, 39: 175-193.
[5] Cremers D, Rousson M, Deriche R. A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape[J]. International Journal of Computer Vision, 2007, 72(2): 195-215.
[6] Kamnitsas K, Ledig C, Newcombe V F, et al. Efficient multi- scale 3D CNN with fully connected CRF for accurate brain lesion segmentation[J]. Medical Image Analysis, 2017, 36: 61-78.
[7] Dolz J, Desrosiers C, Ayed I B. 3D fully convolutional net-works for subcortical segmentation in MRI: a large-scale study[J]. NeuroImage, 2018, 170: 456-470.
[8] Menze B H, Jakab A, Bauer S, et al. The multimodal brain tumor image segmentation benchmark (BRATS)[J]. IEEE Transactions on Medical Imaging, 2015, 34(10): 1993-2024.
[9] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 39(4): 640-651.
[10] Ronneberger O, Fischer P, Brox T. U-Net: convolutional net-works for biomedical image segmentation[C]//LNCS 9351: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Oct 5-9, 2015. Berlin, Heidelberg: Springer, 2015: 234-241.
[11] Roth H R, Lu L, Farag A, et al. DeepOrgan: multi-level deep convolutional networks for automated pancreas segmenta-tion[C]//LNCS 9349: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Oct 5-9, 2015. Berlin, Heidel-berg: Springer, 2015: 556-564.
[12] Dou Q, Chen H, Yu L, et al. Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1182-1195.
[13] Khened M, Alex V, Krishnamurthi G. Fully convolutional multi-scale residual densenets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers[J]. Medical Image Analysis, 2018, 51: 21-45.
[14] Roth H R, Lu L, Lay N, et al. Spatial aggregation of holistically-nested convolutional neural networks for automated pancreas localization and segmentation[J]. Medical Image Analysis, 2018, 45: 94-107.
[15] Liao F, Liang M, Li Z, et al. Evaluate the malignancy of pul-monary nodules using the 3D deep leaky noisy-or network[J]. arXiv:1711.08324, 2017.
[16] Schlemper J, Oktay O, Chen L, et al. Attention-gated networks for improving ultrasound scan plane detection[J]. arXiv:1804. 05338, 2018.
[17] Oktay O, Schlemper J, Folgoc L L, et al. Attention U-Net: learning where to look for the pancreas[J]. arXiv:1804.03999, 2018.
[18] Çiçek Ö, Abdulkadir A, Lienkamp S S, et al. 3D U-Net: lear-ning dense volumetric segmentation from sparse annotation[C]//LNCS 9901: Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, Athens, Oct 17-21, 2016. Berlin, Heid-elberg: Springer, 2016: 424-432.
[19] Milletari F, Navab N, Ahmadi S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]//Proceedings of the 4th International Conference on 3D Vision, Stanford, Oct 25-28, 2016. Washington: IEEE Com-puter Society, 2016: 565-571.
[20] He K M, Zhang X Y, Ren S Q, et al. Delving deep into recti-fiers: surpassing human-level performance on ImageNet classi-fication[C]//Proceedings of the 2015 International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 1026-1034.
[21] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Con-ference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[22] Springenberg J T, Dosovitskiy A, Brox T, et al. Striving for simplicity: the all convolutional net[J]. arXiv:1412.6806, 2015.
[23] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Pro-ceedings of the 32nd International Conference on Machine Learning, Lille, Jul 6-11, 2015: 448-456.
[24] Roth H R, Oda H, Hayashi Y, et al. Hierarchical 3D fully con-volutional networks for multi-organ segmentation[J]. arXiv:1704.06382, 2017.
[25] Wang F, Jiang M Q, Qian C, et al. Residual attention network for image classification[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 3156-3164.
[26] Luong M T, Pham H, Manning C D. Effective approaches to attention-based neural machine translation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Sep 17-21, 2015. Stroudsburg: ACL, 2015: 1412-1421.
[27] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate[J]. arXiv:1409.0473, 2014.
[28] Wang X L, Girshick R, Gupta A, et al. Non-local neural networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 7794-7803.
[29] Osher S, Sethian J A. Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formu-lations[J]. Journal of Computation Physics, 1988, 79(1): 12-49.
[30] Chollet F. Keras[EB/OL]. (2019-11-06)[2019-07-04]. https://github.com/fchollet/keras.
[31] Kingma D P, Ba J. Adam: a method for stochastic optimi-zation[J]. arXiv:1412.6980, 2014.
[32] Wang Y, Liatsis P. An automated method for segmentation of coronary arteries in coronary CT imaging[C]//Proceedings of the 2010 International Conference on Developments in E-systems Engineering, London, Sep 6-8, 2010. Piscataway: IEEE, 2010: 12-16.