Review of Transfer Learning Applied to Diagnosis of COVID-19

doi:10.3778/j.issn.1673-9418.2207080

Abstract

Abstract: Since the outbreak of the coronavirus disease 2019 (COVID-19) epidemic, the number of infections and deaths caused by the epidemic has continued to increase due to its highly contagious nature. The best way to prevent outbreaks of COVID-19 in society is to detect it early in infection through methods such as reverse transcription-polymerase chain reaction (RT-PCR) test and manual examination. However, such laboratory procedures are inefficient, and radiologists’ diagnosis of X-ray and CT images is time-consuming and prone to diagnostic errors. Researchers have proposed computer-aided diagnostic algorithms based on transfer learning, which can minimize the problems arising from traditional diagnostic methods. But there are few reviews on the application of transfer learning in the imaging of COVID-19. Therefore, this paper summarizes and analyzes the current research results on the diagnosis of COVID-19 based on transfer learning techniques at home and abroad. The model types are categorized and discussed, and six perspectives of dataset sources, data preprocessing methods, transfer learning based diagnostic models, model visualization, evaluation metrics, and model performance are analyzed and compared, respectively. The current challenges and future development directions are also pointed out, laying foundation for further research in the future.

Key words: coronavirus disease 2019 (COVID-19), transfer learning, computer-aided diagnosis, medical image processing

摘要： 新型冠状病毒肺炎（COVID-19）疫情爆发以来，由于该病毒具有极强的传染性，所导致的感染人数与死亡人数持续增加。筛查疑似患者和早期诊断COVID-19是防止疫情恶化的重要措施之一。通过核酸检测和人工检查等方法在感染早期诊断出COVID-19是防止其在社会中爆发的最佳途径。然而核酸检测效率低下，仅仅依靠放射科专家诊断X射线图像和CT扫描图像存在耗时长且易出现诊断误差等问题。研究人员相继提出了基于迁移学习的计算机辅助诊断算法，可以最大程度地减少传统诊断方法所产生的问题，但目前关于迁移学习在新冠肺炎成像中的应用综述较少，因此总结和分析了当前国内外基于迁移学习技术诊断COVID-19的研究成果。针对模型类型进行分类讨论，分别从数据集来源、数据预处理方法、基于迁移学习的诊断模型、模型可视化、评价指标以及模型性能6个角度进行分析和比较。并指出了当前所面临的挑战和未来的发展方向，为今后进一步的研究工作奠定了基础。

关键词: 新型冠状病毒肺炎（COVID-19）, 迁移学习, 计算机辅助诊断, 医学影像处理

MENG Wei, YUAN Yilin. Review of Transfer Learning Applied to Diagnosis of COVID-19[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(3): 561-576.

孟伟, 袁艺琳. 迁移学习应用于新型冠状病毒肺炎诊断综述[J]. 计算机科学与探索, 2023, 17(3): 561-576.

References

[1] CHAN J F, YUAN S, KOK K H, et al. A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster[J]. Lancet, 2020, 395: 514-523.
[2] 吕晓亚, 宋磊, 马昭, 等. 新型冠状病毒疫苗免疫策略的强化和优化[J]. 上海交通大学学报(医学版), 2021, 41(12): 1545-1550.
LV X Y, SONG L, MA Z, et al. Strengthening and optimization of immunization strategy for COVID-19 vaccines[J]. Journal of Shanghai Jiao Tong University (Medical Science), 2021, 41(12): 1545-1550.
[3] SINGHAL T. A review of coronavirus disease-2019 (COVID-19)[J]. Indian Journal of Pediatrics, 2020, 87(4): 281-286.
[4] WIKRAMARATNA P, PATON R S, GHAFARI M, et al. Estimating false-negative detection rate of SARS-CoV-2 by RT-PCR [EB/OL]. [2020-04-14]. https://www.medrxiv.org/content/10.1101/2020.04.05.20053355v2.
[5] XIE X, ZHONG Z, ZHAO W, et al. Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing[J]. Radiology, 2020, 296(2): 200343.
[6] 李振昊, 高小玲, 杨小娟, 等. 新型冠状病毒核酸检测分析[J]. 检验医学与临床, 2020, 17(10): 1313-1315.
LI Z H, GAO X L, YANG X J, et al. Nucleic acid detection and analysis of COVID-19[J]. Laboratory Medicine and Clinic, 2020, 17(10): 1313-1315.
[7] GUAN W J, NI Z Y, HU Y, et al. Clinical characteristics of coronavirus disease 2019 in China[J].?The New England Journal of Medicine, 2020, 382(18): 1708-1720.
[8] HUANG C, WANG Y, LI X, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China[J]. Lancet, 2020, 395: 497-506.
[9] 赵张平, 朱永高, 杨燕, 等. 新冠肺炎初诊影像学表现与特征浅析[J]. 罕少疾病杂志, 2021, 28(5): 27-30.
ZHAO Z P, ZHU Y G, YANG Y, et al. The radiographic findings and features of initial HRCT for corona virus disease 2019(COVID-19)[J]. Journal of Rare and Uncommon Diseases, 2021, 28(5): 27-30.
[10] CHUNG M, BERNHEIM A, MEI X, et al. CT imaging features of 2019 novel coronavirus (2019-nCoV)[J]. Radiology, 2020, 295(1): 202-207.
[11] LI Y, XIA L. Coronavirus disease 2019 (COVID-19): role of chest CT in diagnosis and management[J]. American Journal of Roentgenology, 2020, 214(6): 1280-1286.
[12] KANNE J P. Chest CT findings in 2019 novel coronavirus (2019-nCoV) infections from Wuhan, China: key points for the radiologist[J]. Radiology, 2020, 295(1):16-17.
[13] BERNHEIM A, MEI X, HUANG M, et al. Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection[J]. Radiology, 2020, 295(3): 200463.
[14] GREENSPAN H, GINNEKEN B, SUMMERS R. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1153-1159.
[15] BAR Y, DIAMANT I, WOLF L, et al. Chest pathology detection using deep learning with non-medical training[C]//Proceedings of the 12th IEEE International Symposium on Biomedical Imaging, Brooklyn, Apr 16-19, 2015. Piscataway: IEEE, 2015: 294-297.
[16] WU K, CHEN X, DING M. Deep learning based classification of focal liver lesions with contrast-enhanced ultrasound[J]. Optik, 2014, 125(15): 4057-4063.
[17] BURLINA P, BILLINGS S, JOSHI N, et al. Automated diagnosis of myositis from muscle ultrasound: exploring the use of machine learning and deep learning methods[J]. PLoS ONE, 2017, 12(8): e0184059.
[18] SHIN H, ROTH H, GAO M, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1285-1298.
[19] SIRINUKUNWATTANA K, SHAN E, TSANG Y W, et al. Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1196-1206.
[20] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Jun 20-23, 2014. Washington: IEEE Computer Society, 2014: 580-587.
[21] DA NóBREGA R V M, PEIXOTO S A, DA SILVA S P P, et al. Lung nodule classification via deep transfer learning in CT lung images[C]//Proceedings of the 31st IEEE Inter-national Symposium on Computer-Based Medical Systems, Karlstad, Jun 18-21, 2018. Washington: IEEE Computer Society, 2018: 244-249.
[22] KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[23] BEHZADI-KHORMOUJI H, ROSTAMI H, SALEHI S, et al. Deep learning, reusable and problem-based architectures for detection of consolidation on chest X-ray images[J]. Computer Methods and Programs in Biomedicine, 2020, 185: 105162.
[24] COHEN J P, MORRISON P, DAO L. COVID-19 image data collection[J]. arXiv:2003.11597, 2020.
[25] KERMANY D S, GOLDBAUM M, CA I W, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning[J]. Cell, 2018, 172(5): 1122-1131.
[26] CHOWDHURY M, RAHMAN T, KHANDAKAR A, et al. Can AI help in screening viral and COVID-19 pneumonia?[J]. IEEE Access, 2020, 8: 132665-132676.
[27] WANG X S, PENG Y F, LU L, et al. ChestX-Ray8: hospital-scale chest X-RAY database and benchmarks on weakly-supervised classification and localization of common thorax diseases[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 3462-3471.
[28] WANG L, LIN Z Q, WONG A. COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images[J]. Scientific Reports, 2020, 10(1): 19549.
[29] HE X, YANG X, ZHANG S, et al. Sample-efficient deep learning for COVID-19 diagnosis based on CT scans[EB/OL]. [2020-04-13]. https://www.medrxiv.org/content/10.1101/2020. 04.13.20063941v1.
[30] ZHANG K, LIU X, SHEN J, et al. Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography[J]. Cell, 2020, 181(6): 1423-1433.
[31] SOARES E, ANGELOV P, BIASO S, et al. SARS-CoV-2 CT-scan dataset: a large dataset of real patients CT scans for SARS-CoV-2 identification[EB/OL]. [2020-05-14]. https://www.medrxiv.org/content/10.1101/2020.04.24.20078584v3.
[32] RAHIMZADEH M, ATTAR A, SAKHAEI S. A fully automated deep learning-based network for detecting COVID-19 from a new and large lung CT scan dataset[J]. Biomedical Signal Processing and Control, 2021, 68: 102588.
[33] MOROZOV S P, ANDREYCHENKO A E, PAVLOV N A, et al. MosMedData: chest CT scans with COVID-19 related findings dataset[J]. arXiv:2005.06465, 2020.
[34] DE LA IGLESIA VAYá M, SABORIT J M, MONTELL J A, et al. BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patients[J]. arXiv:2006.01174, 2020.
[35] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems 27, Montreal, Dec 8-13, 2014. Cambridge: MIT Press, 2014: 2672-2680.
[36] AYANA G, DESE K, CHOE S W. Transfer learning in breast cancer diagnoses via ultrasound imaging[J]. Cancers, 2021, 13(4): 738.
[37] SHEYKHIVAND S, MOUSAVI Z, MOJTAHEDI S, et al. Developing an efficient deep neural network for automatic detection of COVID-19 using chest X-ray images[J]. Ale-xandria Engineering Journal, 2021, 60(3): 2885-2903.
[38] GIFANI P, SHALBAF A, VAFAEEZADEH M. Automated detection of COVID-19 using ensemble of transfer learning with deep convolutional neural network based on CT scans[J]. International Journal of Computer Assisted Radiology and Surgery, 2021, 16(1): 115-123.
[39] AL-KARAWI D, AL-ZAIDI S, POLUS N, et al. Machine learning analysis of chest CT scan images as a comp-lementary digital test of coronavirus (COVID-19) patients[EB/OL]. [2020-04-17]. https://www.medrxiv.org/content/10. 1101/2020.04.13.20063479v1.
[40] LV D, QI W T, LI Y X, et al. A cascade network for detecting COVID-19 using chestx-rays [J]. arXiv:2005.01468, 2020.
[41] TAJBAKHSH N, SHIN J Y, GURUDU S R, et al. Con-volutional neural networks for medical image analysis: full training or fine tuning?[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1299-1312.
[42] WEISS K, KHOSHGOFTAAR T M, WANG D D. A survey of transfer learning[J]. Journal of Big Data, 2016, 3(1): 9.
[43] LI Z, HOIEM D. Learning without forgetting[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(12): 2935-2947.
[44] RAHAMAN M M, LI C, YAO Y, et al. Identification of COVID-19 samples from chest X-Ray images using deep learning: a comparison of transfer learning approaches[J]. Journal of X-Ray Science and Technology, 2020, 28(5): 821-839.
[45] ZHANG Y D, SATAPATHY S C, ZHANG X, et al. COVID-19 diagnosis via DenseNet and optimization of transfer learning setting[J]. Cognitive Computation, 2021(5): 1-17.
[46] KUMAR N, GUPTA M, GUPTA D, et al. Novel deep transfer learning model for COVID-19 patient detection using X-ray chest images[J]. Journal of Ambient Intelligence and Humanized Computing , 2021, 11(1): 1-10.
[47] RAJARAMAN S, SIEGELMAN J, ALDERSON P O, et al. Iteratively pruned deep learning ensembles for COVID-19 detection in chest X-rays[J]. IEEE Access, 2020, 8: 115041-115050.
[48] KUNDU R, SINGH P K, MIRJALILI S, et al. COVID-19 detection from lung CT-Scans using a fuzzy integral-based CNN ensemble[J]. Computers in Biology and Medicine, 2021, 138: 104895.
[49] PAUL A, BASU A, MAHMUD M, et al. Inverted bell-curve-based ensemble of deep learning models for detection of COVID-19 from chest X-rays[J]. Neural Computing and Applications, 2022, 30: 1-15.
[50] SAMSON A, ANNAVARAPU C. Deep learning-based improved snapshot ensemble technique for COVID-19 chest X-ray classification[J]. Applied Intelligence, 2021, 51(4): 3104-3120.
[51] NIU S, LIU M, LIU Y, et al. Distant domain transfer learning for medical imaging[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(10): 3784-3793.
[52] PERUMAL V, NARAYANAN V, RAJASEKAR S. Detection of COVID-19 using CXR and CT images using transfer learning and haralick features[J]. Applied Intelligence, 2021, 51(1): 341-358.
[53] UM A, MZH A, MOA B, et al. SAM: self-augmentation mechanism for COVID-19 detection using chest X-ray images[J]. Knowledge-Based Systems, 2022, 241: 108207.
[54] LE Q V, KARPENKO A, NGIAM J, et al. ICA with reconstruction cost for efficient overcomplete feature learning[C]//Proceedings of the 25th Annual Conference on Neural Information Processing Systems 2011, Granada, Dec 12-14,2011. Red Hook: Curran Associates, 2011: 1017-1025.
[55] JOKANDAN A S, ASGHARNEZHAD H, JOKANDAN S S, et al. An uncertainty-aware transfer learning-based framework for COVID-19 diagnosis[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(4): 1408-1417.
[56] ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 2921-2929.
[57] CHATTOPADHAY A, SARKAR A, HOWLADER P, et al.Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks[C]//Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision, Nevada, Mar 12-25, 2018. Washington: IEEE Computer Society, 2018: 839-847.
[58] BACH S, BINDER A, MONTAVON G, et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation[J]. PLoS One, 2015, 10(7):e0130140.
[59] RIBEIRO M T, SINGH S, GUESTRIN C. “Why should I trust you?” Explaining the predictions of any classifier[C]//Proceedings of the 22nd ACM SIGKDD International Con-ference on Knowledge Discovery and Data Mining, San Fran-cisco, Aug 13-17, 2016. New York: ACM, 2016: 1135-1144.
[60] HILMIZEN N, BUSTAMAM A, SARWINDA D. The multi-modal deep learning for diagnosing COVID-19 pneumonia from chest CT-scan and X-ray images[C]//Proceedings of the 3rd International Seminar on Research of Information Technology and Intelligent Systems, Yogyakarta, Dec 10-11, 2020. Piscataway: IEEE, 2020: 26-31.
[61] ABDAR M, SAMAMI M, MAHMOODABAD S D, et al. Uncertainty quantification in skin cancer classification using three-way decision-based Bayesian deep learning[J]. Computers in Biology and Medicine, 2021, 135(10): 104418.
[62] JUAN M, ROBERTO P, DANIEL R. Calibration of deep probabilistic models with decoupled Bayesian neural networks[J]. Neurocomputing, 2020, 407: 194-205.
[63] NEAL R M. Bayesian learning for neural networks[M]. Berlin, Heidelberg: Springer, 1996.
[64] KUPINSKI M A, HOPPIN J W, CLARKSON E, et al. Ideal-observer computation in medical imaging with use of Markov-chain Monte Carlo techniques[J]. Journal of the Optical Society of America A Optics Image Science & Vision, 2003, 20(3): 430-438.
[65] GAL Y, GHAHRAMANI Z. Dropout as a Bayesian appro-ximation: representing model uncertainty in deep learning[C]//Proceedings of the 33rd International Conference on Machine Learning, New York, Jun 19-24, 2016. New York: ACM, 2016: 1050-1059.
[66] CHEN T Q, FOX E B, GUESTRIN C. Stochastic gradient Hamiltonian Monte Carlo[C]//Proceedings of the 2014 Interna-tional Conference on Machine Learning, Beijing, Jun 21-26, 2014. New York: ACM, 2014: 1683-1691.