Classification of Pulmonary Nodules Based on CNN Multi-level Second-order Feature Fusion

doi:10.3778/j.issn.1673-9418.1908002

Abstract

Abstract:

The CT image of the lung has a large number of slices. The positions and shapes of the lung nodules in the CT image are different, and the environment around the lung nodules is complex. Traditional methods usually detect pulmonary nodules only by the shape, gray scale and other characteristics, with low utilization rate of pul-monary nodules characteristic information, and do not fully consider the fine-grained characteristics of lung nodules. This paper designs a multi-level second-order feature fusion model based on convolutional neural network (CMSFF). The convolution neural network is used to extract the features of the multi-level sections of the same nodule. Through the two-stage feature fusion, the fine-grained features of the pulmonary nodules are fully considered to achieve accurate extraction of the lung nodule feature information. Experimental results show that this paper reaches an AUC value of 0.924 in the classification of malignant degree of pulmonary nodules through the characteristic information extracted by this method which can effectively improve the classification accuracy of malignant degree of pulmonary nodules.

Key words: CT image, pulmonary nodule, feature extraction, convolutional neural network (CNN)

摘要：

肺部CT图像具有切片数量巨大，肺结节在图像中的位置和形状各异，且肺结节周围环境复杂等特点，传统肺结节检测方法通常只利用肺结节形状、灰度等特征，肺结节特征信息利用率低，且没有完全考虑肺结节细粒度特征信息。为此提出了基于卷积神经网络多层面二阶特征融合模型（CMSFF）。采用卷积神经网络对同一结节的多层切面分别进行特征提取，通过两个阶段的特征融合，充分考虑肺结节的细粒度特征，实现对肺结节特征信息的准确提取。实验表明，该方法提取到的肺结节特征信息在肺结节恶性程度分类中AUC值达到0.924，能有效提高肺结节恶性程度分类准确率。

关键词: CT图像, 肺结节, 特征提取, 卷积神经网络（CNN）

LI Wei, ZHAO Xiaole, DUAN Yanlong, LIU Lijun, HUANG Qingsong. Classification of Pulmonary Nodules Based on CNN Multi-level Second-order Feature Fusion[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(9): 1590-1601.

李维，赵晓乐，段彦隆，刘利军，黄青松. 基于CNN多层面二阶特征融合的肺结节分类[J]. 计算机科学与探索, 2020, 14(9): 1590-1601.

References

[1] Liu X L, Hou F, Qin H, et al. A CADe system for nodule detection in thoracic CT images based on artificial neural network[J]. Science China Information Sciences, 2017, 60(7): 072106.
[2] Zheng G Y, Liu X B, Han G H. Survey on medical image computer aided detection and diagnosis systems[J]. Journal of Software, 2018, 29(5): 1471-1514. 郑光远, 刘峡壁, 韩光辉. 医学影像计算机辅助检测与诊断系统综述[J]. 软件学报, 2018, 29(5): 1471-1514.
[3] Chen S Y, Chao Y, Zou L. Detection of solitary pulmonary nodules based on geometric features[J]. Journal of Biomedical Engineering, 2016, 33(4): 680-685. 陈树越, 晁亚, 邹凌. 基于几何特征的孤立性肺结节检测[J]. 生物医学工程学杂志, 2016, 33(4): 680-685.
[4] van Ginneken B, Iii S G A, Hoop B D, et al. Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: the ANODE09 study[J]. Medical Image Analysis, 2010, 14(6): 707-722.
[5] Greenspan H, Ginneken B, Summers R M. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1153-1159.
[6] van Ginneken B, Setio A A A, Jacobs C, et al. Off-the-shelf convolutional neural network features for pulmonary nodule detection in computed tomography scans[C]//Proceedings of the 12th International Symposium on Biomedical Imaging, Brooklyn, Apr 16-19, 2015. Piscataway: IEEE, 2015: 286-289.
[7] Anthimopoulos M, Christodoulidis S, Ebner L, et al. Lung pattern classification for interstitial lung diseases using a deep convolutional neural network[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1207-1216.
[8] Setio A A A, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1160-1169.
[9] Dou Q, Chen H, Jin Y M, et al. Automated pulmonary nodule detection via 3D ConvNets with online sample filtering and hybrid-loss residual learning[C]//LNCS 10435: Proceedings of the 20th International Conference on Medical Image Com-puting & Computer-Assisted Intervention, Quebec City, Sep 11-13, 2017. Berlin, Heidelberg: Springer, 2017: 630-638.
[10] Wu C R, Jie B, Ye M Q. Reviews on computer-aided detection and diagnosis of pulmonary nodules in CT images[J]. Data Acquisition and Processing, 2016, 31(5): 868-881. 伍长荣, 接标, 叶明全. CT图像肺结节计算机辅助检测与诊断技术研究综述[J]. 数据采集与处理, 2016, 31(5): 868-881.
[11] Sato Y, Nakajima S, Shiraga N, et al. 3D multi-scale line filter for segmentation and visualization of curvilinear struc-tures in medical images[C]//LNCS 1205: Proceedings of the 1st Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine and Medial Robotics and Computer-Assisted Surgery, Grenoble, Mar 19-22, 1997. Berlin, Hei-delberg: Springer, 1997: 213-222.
[12] Yankelevitz D F, Reeves A P, Kostis W J, et al. Small pul-monary nodules: volumetrically determined growth rates based on CT evaluation[J]. Radiology, 2000, 217(1): 251-256.
[13] Kostis W J, Reeves A P, Yankelevitz D F, et al. Three-dimensional segmentation and growth-rate estimation of small pulmonary nodules in helical CT images[J]. IEEE Transac-tions on Medical Imaging, 2003, 22(10): 1259-1274.
[14] Dehmeshki J, Amin H, Valdivieso M, et al. Segmentation of pulmonary nodules in thoracic CT scans: a region growing approach[J]. IEEE Transactions on Medical Imaging, 2008, 27(4): 467-480.
[15] Way T W, Sahiner Bchan H P. Computer-aided diagnosis of pulmonary nodules on CT scans: improvement of classifica-tion performance with nodule surface features[J]. Medical Physics, 2009, 36(7): 3086-3098.
[16] Orozco H M, Villegas O O V, Sánchez V G C, et al. Auto-mated system for lung nodules classification based on wavelet feature descriptor and support vector machine[J]. BioMedical Engineering OnLine, 2015, 14(1): 9.
[17] Firmino M, Angelo G, Morais H, et al. Computer-aided detection (CADe) and diagnosis (CADx) system for lung cancer with likelihood of malignancy[J]. BioMedical Engin-eering OnLine, 2016, 15(1): 2.
[18] Francesco C, Hoop B D, Riel S V, et al. Automatic class-ification of pulmonary peri-fissural nodules in computed to-mography using an ensemble of 2D views and a convolu-tional neural network out-of-the-box[J]. Medical Image Analysis, 2015, 26(1): 195-202.
[19] Yang F, Xie H W, Liu A Y. Lung nodule classification algorithm based on convolutional neural network[J]. Computer Engineering and Applications, 2019, 55(7): 145-150. 杨帆, 谢红薇, 刘爱媛. 基于卷积神经网络的肺结节分类算法[J]. 计算机工程与应用, 2019, 55(7): 145-150.
[20] Pinheiro C A D P, Nedjah N, Mourelle L D M. Detection and classification of pulmonary nodules using deep learning and swarm intelligence[J]. Multimedia Tools and Applications, 2019(2): 1-29.
[21] Zhang G, Zhu D, Liu X, et al. Multi-scale pulmonary nodule classification with deep feature fusion via residual network[J]. Journal of Ambient Intelligence and Humanized Computing, 2018.
[22] Karpathy A, Joulin A, Li F F. Deep fragment embeddings for bidirectional image sentence mapping[C]//Proceedings of the 2014 International Conference on Neural Information Processing Systems, Montreal, Nov 3-6, 2014. Cambridge: MIT Press, 2014: 1889-1897.
[23] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[J]. arXiv:1505. 04597, 2015.
[24] Gillies R J, Kinahan P E, Hricak H. Radiomics: images are more than pictures, they are data[J]. Radiology, 2015, 278(2): 563-577.
[25] Lin T Y, Dollár P, Girshick R B, et al. Feature pyramid networks for object detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 936-944.
[26] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[27] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classifi-cation with deep convolutional neural networks[J]. Com-munications of the ACM, 2017, 60(6): 84-90.
[28] Armato S G, Hadjiiski L, Tourassi G D, et al. Guest editorial: LUNGx challenge for computerized lung nodule classifica-tion: reflections and lessons learned[J]. Journal of Medical Imaging, 2015, 2(2): 020103.
[29] Chen S. Pre-diagnosis research of lung cancer based on CT image[D]. Chengdu: University of Electronic Science and Technology of China, 2018. 陈实. 基于CT图像的肺癌前期辅助诊断研究[D]. 成都: 电子科技大学, 2018.
[30] Wang H H. Deep learning based pulmonary nodule detection algorithms for medical images[D]. Xi??an: Xidian University, 2018. 王厚华. 基于深度学习的医学图像肺结节检测算法研究[D]. 西安: 西安电子科技大学, 2018.