Review of Deep Learning in Classification of Tongue Image

doi:10.3778/j.issn.1673-9418.2208052

Abstract

Abstract: With the rapid development of technology and the improvement of computing power, deep learning will be widely used in the field of tongue classification. The classification of tongue image is an important part of tongue diagnosis in traditional Chinese medicine (TCM). Traditional tongue diagnosis is dependent on understanding and judgment skills gained from personal experience under the guidance of basic theory, which leads to ambiguity and variability, affecting diagnostic reproducibility. In order to reduce the error of subjective judgment, many researchers have devoted themselves to realizing the objectification, quantification and automation of tongue diagnosis in TCM through deep learning. This paper analyzes and summarizes the research status of tongue image classification methods based on deep learning. In the study of tongue image classification, various deep learning methods are used as the research objects. The research objects are divided into categories based on early neural networks, convolutional neural networks, regional convolutional neural networks, transfer learning and other methods for summary analysis. TCM syndromes/diseases in tongue diagnosis and classification of physical constitution are discussed. A 5-fold cross-validation experiment is conducted with the public tongue diagnosis dataset on Kaggle. The dataset is a small sample of the dentate tongue, and the classification methods based on deep learning and transfer learning are evaluated. The research development of single-label and multi-label classification is discussed and prospected.

Key words: traditional Chinese medicine tongue diagnosis, deep learning, tongue image analysis, tongue image classification

摘要： 随着技术的快速发展和计算能力的提高，深度学习在舌象分类领域将得到广泛的应用。舌诊图像的舌象分类是中医舌诊客观化的重要组成部分。传统的舌诊是在基础理论的指导下，借助个人经验所做出的理解和判断，因而会具有一定的差异性和模糊性，影响诊断的可重复性。为了减少主观判断的误差，许多研究人员致力于通过深度学习实现中医舌诊的客观化、定量化和自动化。主要对基于深度学习的舌象分类方法研究现状进行分析梳理和归纳总结。在舌象分类研究中，以各类深度学习方法作为研究对象，将其划分为基于早期神经网络、卷积神经网络、区域卷积神经网络、迁移学习以及其他方法进行总结分析；对舌诊中的中医证候和疾病以及体质分类进行了讨论；用Kaggle上的公开舌诊数据集进行5折交叉验证实验，数据集为小样本齿痕舌，评估了基于深度学习和迁移学习分类方法；对舌诊图像质量、构建数据集方式、特征提取、单标签和多标签分类的研究发展进行了探讨和展望。

关键词: 中医舌诊, 深度学习, 舌象分析, 舌象分类

WU Xin, XU Hong, LIN Zhuosheng, LI Shengke, LIU Huilin, FENG Yue. Review of Deep Learning in Classification of Tongue Image[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(2): 303-323.

吴欣, 徐红, 林卓胜, 李胜可, 刘慧琳, 冯跃. 深度学习在舌象分类中的研究综述[J]. 计算机科学与探索, 2023, 17(2): 303-323.

References

[1] MACIOCIA G. Tongue diagnosis in Chinese medicine [M]. Seattle: Eastland Press, 1995.
[2] ZHANG D. Automated biometrics: technnologies and systems[M]. Berlin, Heidelberg: Springer, 2000.
[3] BARBARA K. Atlas of Chinese of tongue diagnosis[M]. Seattle: Eastland Press, 1998.
[4] PANG B, ZHANG D, LI N, et al. Computerized tongue diagnosis based on Bayesian networks[J]. IEEE Transactions on Biomedical Engineering, 2004, 51(10): 1803-1810.
[5] ZHI L, ZHANG D, YAN J Q, et al. Classification of hyper-spectral medical tongue images for tongue diagnosis[J]. Computerized Medical Imaging and Graphics, 2007, 31(8): 672-678.
[6] QI Z, TU L P, CHEN J B, et al. The classification of tongue colors with standardized acquisition and ICC profile correction in traditional Chinese medicine[J]. BioMed Research International, 2016: 3510807.
[7] 王昇, 刘开华, 王丽婷. 舌诊图像点刺和瘀点的识别与提取[J]. 计算机工程与科学, 2017, 39(6): 1126-1132.
WANG S, LIU K H, WANG L T. Tongue spots and petechiae recognition and extraction in tongue diagnosis images[J]. Computer Engineering & Science, 2017, 39(6): 1126-1132.
[8] YAMAMOTO S, TSUMURA N, NAKAGUCHI T, et al. Principal component vector rotation of the tongue color spectrum to predict “Mibyou”(disease-oriented state)[J]. International Journal of Computer Assisted Radiology and Surgery, 2011, 6(2): 209-215.
[9] 刘畅. 基于决策树及神经网络对儿童抽动障碍肾志不足证舌象的研究[D]. 济南: 山东中医药大学, 2020.
LIU C. A study of tongue feature in children with tic disorders of kidney emotion deficiency based on decision tree and neural network[D]. Jinan: Shandong University of Traditional Chinese Medicine, 2020.
[10] 王奕然, 张新峰. 基于 AdaBoost 级联框架的舌色分类[J]. 北京生物医学工程, 2020, 39(1): 8-14.
WANG Y R, ZHANG X F. Tongue color classification based on AdaBoost cascade framework[J]. Beijing Biomedical Engineering, 2020, 39(1): 8-14.
[11] BHARATHI M, PRASAD D, VENKATAKRISHNAMOOR-THY T, et al. Diabetes diagnostic method based on tongue image classification using machine learning algorithms[J]. Journal of Pharmaceutical Negative Results, 2022, 13(4): 1247-1250.
[12] GORUR K, BOZKURT M R, BASCIL M S, et al. GKP signal processing using deep CNN and SVM for tongue-machine interface[J]. Traitement du Signal, 2019, 36(4): 319-329 .
[13] OBAFEMI-AJAYI T, KANAWONG R, XU D, et al. Features for automated tongue image shape classification[C]//Proceedings of the 2012 IEEE International Conference on Bioinformatics and Biomedicine Workshops, Philadelphia, Oct 4-7, 2012. Washington: IEEE Computer Society, 2012: 273-279.
[14] 李宗润. 基于深度学习技术的舌体分割模型研究与舌象智能化应用探索[D]. 成都: 成都中医药大学, 2020.
LI Z R. The study of tongue segmentation model based on deep learning technology and application exploration of the intelligence of tongue image[D]. Chengdu: Chengdu University of Traditional Chinese Medicine, 2020.
[15] 李红岩, 李灿, 郎许锋,等. 中医四诊智能化研究现状及热点分析[J]. 南京中医药大学学报, 2022, 38(2): 180-186.
LI H Y, LI C, LANG X F, et al. Analysis of the research status and hot spot of intelligent four-diagnosis in TCM[J]. Journal of Nanjing University of Traditional Chinese Medicine, 2022, 38(2): 180-186.
[16] 牛富泉. 基于深度学习的中医舌像分类模型研究[D]. 成都: 电子科技大学, 2021.
NIU F Q. Research on tongue image classification in traditional Chinese medicine based on deep learning[D]. Chengdu: University of Electronic Science and Technology of China, 2021.
[17] 鲍康. 基于舌头检测与舌象识别的中医智能诊断系统的设计与实现[D]. 北京: 北京交通大学, 2021.
BAO K. Design and implementation of TCM intelligent diagnosis system based on tongue detection and tongue manifestation recognition[D]. Beijing: Beijing Jiaotong University, 2021.
[18] 刘飞, 张俊然, 杨豪. 基于深度学习的糖尿病患者的分类识别 [J]. 计算机应用, 2018, 38(S1): 39-43.
LIU F, ZHANG J R, YANG H. Classification and recognition of diabetes mellitus based on deep learning[J]. Journal of Computer Application, 2018, 38(S1): 39-43.
[19] SUN Y, DAI S, LI J, et al. Tooth-marked tongue recognition using gradient-weighted class activation maps[J]. Future Internet, 2019, 11(2): 45.
[20] 王东军, 朱青青, 余静寅, 等. 基于CiteSpace的舌诊客观化研究领域科学知识图谱与可视化分析[J]. 世界科学技术: 中医药现代化, 2021, 23(1): 283-291.
WANG D J, LI Q Q, YU J Y, et al. Atlas and visualization analysis of scientific knowledge in the field of tongue objectification research based on CiteSpace[J]. World Science and Technology: Modernization of Traditional Chinese Medicine, 2021, 23(1): 283-291.
[21] 李渊彤, 罗裕升, 朱珍民. 基于深度学习的舌象特征分析 [J]. 计算机科学, 2020, 47(11): 148-158.
LI Y T, LUO Y S, ZHU Z M. Tongue image analysis in traditional Chinese medicine based on deep learning[J]. Computer Science, 2020, 47(11): 148-158.
[22] MCCULLOCH W S, PITTS W. A logical calculus of the ideas immanent in nervous activity[J]. The Bulletin of Mathematical Biophysics, 1943, 5(4): 115-133.
[23] CHEN L, LI S, BAI Q, et al. Review of image classification algorithms based on convolutional neural networks[J]. Remote Sensing, 2021, 13(22): 4712.
[24] ROSENBLATT F. The perceptron: a probabilistic model for information storage and organization in the brain[J]. Psych-ological Review, 1958, 65(6): 386.
[25] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[26] 邢甜甜. 基于卷积神经网络的舌象模式分类研究[D]. 秦皇岛: 燕山大学, 2018.
XING T T. Classification of tongue pattern based on convolution neural network[D]. Qinhuangdao: Yanshan University, 2018.
[27] ZHOU L, PAN S, WANG J, et al. Machine learning on big data: opportunities and challenges[J]. Neurocomputing, 2017, 237: 350-361.
[28] MANGASARIAN O L, MUSICANT D R. Data discrimination via nonlinear generalized support vector machines[M]. Berlin, Heidelberg: Springer, 2001.
[29] 张珂, 冯晓晗, 郭玉荣, 等. 图像分类的深度卷积神经网络模型综述[J]. 中国图象图形学报, 2021, 26(10): 2305-2325.
ZHANG K, FENG X H, GUO Y R, et al. Overview of deep convolutional neural networks for image classification[J]. Journal of Image and Graphics , 2021, 26(10): 2305-2325.
[30] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[31] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//LNCS 8689: Proceedings of the 13th European Conference on Computer Vision, Zurich, Sep 6-12, 2014. Cham: Springer, 2014: 818-833.
[32] XUE Y, LI X, CUI Q, et al. Cracked tongue recognition based on deep features and multiple-instance SVM[C]//LNCS 11165: Proceedings of the 19th Pacific-Rim Conference on Multimedia, Hefei, Sep 21-22, 2018. Cham: Springer,2018: 642-652.
[33] HUO C M, ZHENG H, SU H Y, et al. Tongue shape classification integrating image preprocessing and convolution neural network[C]//Proceedings of the 2017 2nd Asia-Pacific Conference on Intelligent Robot Systems, Wuhan, Jun 16-18, 2017. Piscataway: IEEE, 2017: 42-46.
[34] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning, Lille, Jul 6-11, 2015: 448-456.
[35] 肖庆新, 张菁, 张辉, 等. 基于轻型卷积神经网络的舌苔颜色分类方法[J]. 测控技术, 2019, 38(3): 26-31.
XIAO Q X, ZHANG J, ZHANG H, et al. Tongue coating color classification based on shallow convolutional neural network[J]. Measurement & Control Technology, 2019, 38(3): 26-31.
[36] JIA Y Q, SHELHAMER E, DONAHUE J, et al. Caffe: convolutional architecture for fast feature embedding[C]//Proceedings of the 2014 ACM International Conference on Multimedia, Orlando, Nov 3-7, 2014. New York: ACM, 2014: 675-678.
[37] HOU J, SU H Y, YAN B, et al. Classification of tongue color based on CNN[C]//Proceedings of the 2017 IEEE 2nd International Conference on Big Data Analysis, Beijing, Mar 10-12, 2017. Piscataway: IEEE, 2017: 725-729.
[38] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409. 1556, 2014.
[39] SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 1-9.
[40] RORA S, BHASKARA A, GE R, et al Provable bounds for learning some deep representations[C]//Proceedings of the 31st International Conference on Machine Learning, Beijing, Jun 21-26, 2014: 584-592.
[41] LIN M, CHEN Q, YAN S. Network in network[J]. arXiv:1312.4400, 2013.
[42] FU S, ZHENG H, YANG Z, et al. Computerized tongue coating nature diagnosis using convolutional neural network[C]//Proceedings of the 2017 IEEE 2nd International Conference on Big Data Analysis, Beijing, Mar 10-12, 2017. Piscataway: IEEE, 2017: 730-734.
[43] OTSU N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62-66.
[44] SMITH A R. Color gamut transform pairs[J]. ACM Sig-graph Computer Graphics, 1978, 12(3): 12-19.
[45] HE K M, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[46] LI J, ZHANG Z, ZHU X, et al. Automatic classification framework of tongue feature based on convolutional neural networks[J]. Micromachines, 2022, 13(4): 501.
[47] RONNEBERGER O, FISCHER P, BROX T. U-Net: conv-olutional networks for biomedical image segmentation[C]//LNCS 9351: Proceedings of the 2015 International Con-ference on Medical Image Computing and Computer-Assisted Intervention, Munich, Oct 5-9, 2015. Cham: Springer, 2015: 234-241.
[48] 邵尤伟. 基于深度学习的智能舌诊方法研究[D]. 厦门: 厦门大学, 2018.
SHAO Y W. Research on intelligent tongue diagnosis based on deep leaning[D]. Xiamen: Xiamen University, 2018.
[49] ZHANG D, ZHANG H Z, ZHANG B. Statistical analysis of tongue color and its applications in diagnosis[M]. Berlin,Heidelberg: Springer, 2017.
[50] CHANG W H, CHU H T, CHANG H H. Tongue fissure visualization with deep learning[C]//Proceedings of the 2018 Conference on Technologies and Applications of Artificial Intelligence, Taichung, China, Nov 30-Dec 2, 2018. Washington: IEEE Computer Society, 2018: 14-17.
[51] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization[C]//Proceedings of the 2017 IEEE Inter-national Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 618-626.
[52] TANG Y, SUN Y, CHIANG J Y, et al. Research on multiple-instance learning for tongue coating classification[J]. IEEE Access, 2021, 9: 66361-66370.
[53] ANDREWS S, TSOCHANTARIDIS I, HOFMANN T. Support vector machines for multiple-instance learning[C]//Proceedings of the 2022 Conference on Neural Information Processing Systems, Vancouver, Dec 9-14, 2002. Cambridge: MIT Press, 2002: 561-568.
[54] 孙萌, 张新峰. 基于TripletLoss损失函数的舌象分类方法研究[J]. 北京生物医学工程, 2020, 39(2): 131-137.
SUN M, ZHANG X F. Tongue image classification based TripletLoss metric[J]. Beijing Biomedical Engineering, 2020, 39(2): 131-137.
[55] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 2261-2269.
[56] 陈慧贞. 基于卷积神经网络的舌象辨识模型应用研究 [D]. 秦皇岛：燕山大学, 2019.
CHEN H Z. Research on application of tongue recognition model based on convolutional neural network[D]. Qinhuangdao: Yanshan University, 2019.
[57] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 7132-7141.
[58] 骆明楠. 面向智能舌诊的注意力卷积网络研究[D]. 广州: 华南理工大学, 2020.
LUO M N. Research on attention convolution networks for intelligent tongue diagnosis[D]. Guangzhou: South China University of Technology, 2020.
[59] GIRSHICK R, DONAHUE J, DARELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Jun 23-28, 2014. Washington: IEEE Computer Society, 2014: 580-587.
[60] AZIZ L, SALAM M S B H, SHEIKH U, et al. Exploring deep learning-based architecture, strategies, applications and current trends in generic object detection: a comprehensive review[J]. IEEE Access, 2020, 8: 170461-170495.
[61] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2015, Montreal, Dec 7-12, 2015: 91-99.
[62] GAVRILESCU R, ZET C, FO?AL?U C, et al. Faster R-CNN: an approach to real-time object detection[C]//Proceedings of the 2018 International Conference and Exposition on Electrical and Power Engineering, Iasi, Oct 18-19, 2018. Piscataway: IEEE, 2018: 165-168.
[63] SHAO Q, LI X, FU Z. Recognition of teeth-marked tongue based on gradient of concave region[C]//Proceedings of the 2014 International Conference on Audio, Language and Image Processing, Shanghai, Jul 7-9, 2014. Piscataway:IEEE, 2014: 968-972.
[64] WANG H, ZHANG X, CAI Y. Research on teeth marks recognition in tongue image[C]//Proceedings of the 2014 International Conference on Medical Biometrics, Shenzhen, May 30-Jun 1, 2014. Piscataway: IEEE, 2014: 80-84.
[65] LI X, ZHANG Y, CUI Q, et al. Tooth-marked tongue recognition using multiple instance learning and CNN features[J]. IEEE Transactions on Cybernetics, 2018, 49(2): 380-387.
[66] GHOLAMI E, TABBAKH S R K, KHEIRABADIH M. Proposing method to increase the detection accuracy of stomach cancer based on colour and lint features of tongue using CNN and SVM[J]. arXiv:2011.09962, 2020.
[67] 杜春慧. 中医舌质特征的机器学习模型研究[D]. 成都：电子科技大学, 2020.
DU C H. Research on machine leaning model of tongue characteristics in traditional Chinese medicine[D]. Chengdu: University of Electronic Science and Technology of China, 2020.
[68] CHEN P, MEN S, LIN H, et al. Detection of local lesions in tongue recognition based on improved Faster R-CNN[C]//Proceedings of the 2021 6th International Conference on Computational Intelligence and Applications, Xiamen, Jun 11-13, 2021. Piscataway: IEEE, 2021: 165-168.
[69] HE K, GKIOXARI G, DOLLáR P, et al. Mask R-CNN[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington:IEEE Computer Society, 2017: 2980-2988.
[70] LI J, CHEN Q, HU X, et al. Establishment of noninvasive diabetes risk prediction model based on tongue features and machine learning techniques[J]. International Journal of Medical Informatics, 2021, 149: 104429.
[71] PENG C D, WANG L, JIANG D M, et al. Establishing and validating a spotted tongue recognition and extraction model based on multiscale convolutional neural network[J]. Digital Chinese Medicine, 2022, 5(1): 49-58.
[72] 刘佳丽, 孙自强. 基于Double-D算法的舌像检测[J]. 计算机工程与设计, 2020, 41(7): 2025-2030.
LIU J L, SUN Z Q. Tongue image detection based on Double-D algorithm[J]. Computer Engineering and Design, 2020, 41(7): 2025-2030.
[73] YEN S J, YE S C, LEE C L. Traditional Chinese medicine online intelligent tongue diagnosis bot[C]//Proceedings of the 2021 International Symposium on Intelligent Signal Pr-ocessing and Communication Systems, Hualien, China,Nov 16-19, 2021. Piscataway: IEEE, 2021: 1-2.
[74] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[J]. arXiv: 2004.10934, 2020.
[75] 颜建军, 李东旭, 郭睿, 等. 基于深度学习和随机森林的齿痕舌分类研究[J]. 中华中医药学刊, 2022, 40(2): 19-22.
YAN J J, LI D X, GUO R, et al. Research on classification of dentate tongue based on deep learning and random forest[J]. Chinese Archives of Traditional Chinese Medicine, 2022, 40(2): 19-22.
[76] WENG H, LI L, LEI H, et al. A weakly supervised tooth-mark and crack detection method in tongue image[J]. Con-currency and Computation: Practice and Experience, 2021, 33(16): e6262.
[77] 高爽, 徐巧枝. 迁移学习方法在医学图像领域的应用综述 [J]. 计算机工程与应用, 2021, 57(24): 39-50.
GAO S, XU Q Z. Review of application of transfer learning in medical image field[J]. Computer Engineering and App-lications, 2021, 57(24): 39-50.
[78] CHEN Z, ZHANG X, HUANG W. The similar sparse do-main adaptation illustrated by the case of TCM tongue inspection[C]//Proceedings of the 2020 IEEE International Conference on Bioinformatics and Biomedicine, Seoul, Dec 16-19, 2020. Piscataway: IEEE, 2020: 1520-1525.
[79] 刘婧玮. 基于卷积神经网络的中医舌象辨识人工智能方法学研究[D]. 北京: 北京中医药大学, 2020.
LIU J W. Artificial intelligence methodology for Chinese medicine tongue recognition based on convolutional neural network[D]. Beijing: Beijing University of Chinese Medicine, 2020.
[80] WANG X, LIU J, WU C, et al. Artificial intelligence in tongue diagnosis: using deep convolutional neural network for recognizing unhealthy tongue with tooth-mark[J]. Com-putational and Structural Biotechnology Journal, 2020, 18: 973-980.
[81] 邱童. 基于深度学习与多特征融合的舌象诊断算法[J]. 现代信息科技, 2020, 4(1): 63-65.
QIU T. Tongue image diagnosis algorithm based on deep learning and multi-feature fusion[J]. Modern Information Technology, 2020, 4(1): 63-65.
[82] HU Y, WEN G, LIAO H, et al. Automatic construction of Chinese herbal prescriptions from tongue images using CNNs and auxiliary latent therapy topics[J]. IEEE Transactions on Cybernetics, 2019, 51(2): 708-721.
[83] SONG C, WANG B, XU J. Classifying tongue images using deep transfer learning[C]//Proceedings of the 2020 5th International Conference on Computational Intelligence and Applications, Beijing, Jun 19-21, 2020. Piscataway: IEEE, 2020: 103-107.
[84] SADASIVAN S, SIVAKUMAR T T, JOSEPH A P, et al. Tongue print identification using deep CNN for forensic analysis[J]. Journal of Intelligent & Fuzzy Systems, 2020, 38(5): 6415-6422.
[85] 杨晶东, 张朋. 基于迁移学习的全连接神经网络舌象分类方法[J]. 第二军医大学学报, 2018, 39(8): 897-902.
YANG J D, ZHANG P. Tongue image classification method based on transfer learning and fully connected neural network[J]. Academic Journal of Naval Medical University, 2018, 39(8): 897-902.
[86] 翟鹏博, 杨浩, 宋婷婷, 等. 融合注意力机制的多阶段舌象分类算法[J]. 计算机工程与设计, 2021, 42(6): 1606-1613.
ZHAI P B, YANG H, SONG T T, et al. Multi-stage image classification algorithm incorporating attention mechanism [J]. Computer Engineering and Design, 2021, 42(6): 1606-1613.
[87] KONG X, RUI Y, DONG X, et al. Tooth-marked tongue recognition based on mask scoring R-CNN[C]//Proceedings of the 2020 2nd International Conference on Artificial Intelligence Technologies and Application, Dalian, Aug 21-23, 2020.
[88] 刘梦, 王曦廷, 周璐, 等. 基于深度学习与迁移学习的中医舌象提取识别研究[J]. 中医杂志, 2019(10): 835-840.
LIU M, WANG X Y, ZHOU L, et al. Study on extraction and recognition of traditional Chinese tongue manifestation: based on deep learning and migration learning[J]. Journal of Traditional Chinese Medicine, 2019(10): 835-840.
[89] XU Q, ZENG Y, TANG W, et al. Multi-task joint learning model for segmenting and classifying tongue images using a deep neural network[J]. IEEE Journal of Bio-Medical and Health Informatics, 2020, 24(9): 2481-2489.
[90] 汤一平, 王丽冉, 何霞, 等. 基于多任务卷积神经网络的舌象分类研究[J]. 计算机科学, 2018, 45(12): 255-261.
TANG Y P, WANG L R, HE X, et al. Classification of tongue image based on multi-task deep convolution neural network[J]. Computer Science, 2018, 45(12): 255-261.
[91] 王爱民, 赵忠旭, 沈兰荪. 中医舌象自动分析中舌色, 苔色分类方法的研究[J]. 北京生物医学工程, 2000, 19(3): 136-141.
WANG A M, ZHAO Z X, SHEN L S. Research on the tongue color classification in automatic analysis of traditional Chinese medicine[J]. Beijing Biomedical Engineering, 2000, 19(3): 136-141.
[92] MIKA S, RATSCH G, WESTON J, et al. Fisher discriminant analysis with kernels[C]//Proceedings of the 1999 IEEE Workshop on Neural Networks for Signal Processing, Madison, Aug 25, 1999. Piscataway: IEEE, 1999: 41-48.
[93] 张思远, 翟宏琛, 梁艳梅, 等. 模糊相关中的加权算法及其在彩色图像检索中的应用[J]. 中国科学: G辑物理学、力学、天文学, 2004, 34(1): 60-68.
ZHANG S Y, ZHAI H C, LIANG Y M, et al. Weighting algorithm in fuzzy correlation and its application in color image retrieval[J]. Scientia Sinica: Physica, Mechanica & Astronomica, 2004, 34(1): 60-68.
[94] QIN J, MEN G J, WU G Z, et al. The retrieval of the medical tongue-coating images using fuzzy CMAC neural network[C]//Proceedings of the 2005 IEEE International Conference on Industrial Technology, Hong Kong, China, Dec 14-17, 2005. Piscataway: IEEE, 2005: 465-468.
[95] QIU T. Tongue identification for small samples based on meta learning[C]//Proceedings of the 2020 International Conference on Computer Information and Big Data App-lications, Guiyang, Apr 17-19, 2020. Piscataway: IEEE, 2020: 295-299.
[96] XIAO Q, ZHANG H, ZHANG J, et al. Texture analysis of tongue coating in traditional Chinese medicine based on transfer learning and multi-model decision[J]. Sensing and Imaging, 2021, 22(1): 1-13.
[97] HU Y, WEN G, LUO M, et al. Fully-channel regional attention network for disease-location recognition with tongue images[J]. Artificial Intelligence in Medicine, 2021, 118: 102110.
[98] XU Y, WEN G, YANG P, et al. Task-coupling elastic learning for physical sign-based medical image classification[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 26(2): 626-637.
[99] DAI Y, WANG G. Analyzing tongue images using a conceptual alignment deep autoencoder[J]. IEEE Access, 2018, 6: 5962- 5972.
[100] MENG D, CAO G, DUAN Y, et al. Tongue images class-ification based on constrained high dispersal network [J]. Evidence-Based Complementary and Alternative Medi- cine, 2017(3): 7452427 .
[101] 卓力, 孙亮亮, 张辉, 等. 有噪声标注情况下的中医舌色分类方法[J]. 电子与信息学报, 2022, 44(1): 89-98.
ZHUO L, SUN L L, ZHANG H, et al. TCM tongue color classification method under noisy labeling[J]. Journal of Electronics & Information Technology, 2022, 44(1): 89-98.
[102] RAJAKUMARAN S, SASIKALA J. Improvement in tongue color image analysis for disease identification using deep learning based depthwise separable convolution model[J]. Indian Journal of Computer Science and Engineering, 2021, 12: 21-34.
[103] HANHUI. Tooth-marked-tongue[EB/OL]. (2020-03-10)[2022-07-25]. http://www.kaggle.com/datasets/clearhanhui/biyesheji.
[104] BioHit. TongueImageDataset[EB/OL]. (2014-09-05) [2022- 07-25]. http://github.com/BioHit/TongeImageDatase.
[105] CHAWLA N V, BOWYER K W, HALL L O, et al. SMOTE: synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16: 321-357.
[106] CHEN Z, ZHANG X, QIU R. Application of artificial intelligence in tongue diagnosis of traditional Chinese medicine: a review[J]. TMR Modern Herbal Medicine, 2021, 4(2): 14-30.
[107] 刘国正. 卷积神经网络在中医舌象分类模型中的应用研究[D]. 长春: 吉林大学, 2018.
LIU G Z. Research on application of traditional Chinese medicine tongue images classification based on CNN[D]. Changchun: Jilin University, 2018.
[108] CHIU C C. A novel approach based on computerized image analysis for traditional Chinese medical diagnosis of the tongue[J]. Computer Methods and Programs in Biomedicine, 2000, 61(2): 77-89.
[109] GUI M, ZHANG X, HU G, et al. A study on tongue image color description based on label distribution learning[C]// Proceedings of the 8th International Conference on Biomedical Engineering and Informatics, Shenyang, Oct 14-16, 2015. Piscataway: IEEE, 2015: 148-152.
[110] ZHANG H, WANG K, ZHANG D, et al. Computer aided tongue diagnosis system[C]//Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, Jan 17-18, 2005. Piscataway: IEEE, 2006: 6754-6757.
[111] HUI S C, HE Y, THACH D T C. Machine learning for tongue diagnosis[C]//Proceedings of the 2007 6th Inter-national Conference on Information, Communications & Signal Processing, Singapore, Dec 10-13, 2007. Piscataway: IEEE, 2007: 1-5.
[112] HU J W, YAN Z Z, JIANG J H. Classification of fissured tongue images using deep neural networks[J]. Technology and Health Care, 2022, 30(S1): 271-283.
[113] HSU P C, WU H K, HUANG Y C, et al. The tongue features associated with type 2 diabetes mellitus[J]. Medicine, 2019, 98(19): e15567.
[114] ZHANG B, KUMAR B V, ZHANG D. Detecting diabetes mellitus and nonproliferative diabetic retinopathy using tongue color, texture, and geometry features[J]. IEEE Transactions on Biomedical Engineering, 2013, 61(2): 491- 501.
[115] LI J, HU X, TU L, et al. Diabetes tongue image class-ification using machine learning and deep learning[R]. Shanghai University of Traditional Chinese Medicine, 2021: 1-20.
[116] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[J]. arXiv:201011929, 2020.
[117] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2017, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 5998-6008.
[118] PRAJAPATI M D, NAMDEV N. Prediction of heart disease risk based on deep learning[J]. International Research Journal of Modernization in Engineering Technology and Science, 2022, 4: 1011-1016.
[119] SU J, LI Z, HUANG M, et al. Triglyceride glucose index for the detection of the severity of coronary artery disease in different glucose metabolic states in patients with coronary heart disease: a RCSCD-TCM study in China[J]. Cardiovascular Diabetology, 2022, 21(1): 1-9.
[120] 叶桦, 冯全生, 严小英, 等. 基于人工神经网络的糖尿病合并冠心病舌脉象证型预测研究[J]. 中华中医药杂志, 2020, 35(10): 5184-5187.
YE H, FENG Q S, YAN X Y, et al. Study of tongue and pulses syndrome prediction of diabetes mellitus combined with coronary heart disease based on artificial neural network[J]. China Journal of Traditional Chinese Medicine and Pharmacy, 2020, 35(10): 5184-5187.
[121] LIANG K, HUANG X, CHEN H, et al. Tongue diagnosis and treatment in traditional Chinese medicine for severe COVID-19: a case report[J]. Annals of Palliative Medicine, 2020, 9(4): 2400-2407.
[122] WANG X, WANG X, LOU Y, et al. Constructing tongue coating recognition model using deep transfer learning to assist syndrome diagnosis and its potential in non-invasive ethnopharmacological evaluation[J]. Journal of Ethnophar-macology, 2022, 285: 114905.
[123] 中华中医药学会. 中医体质分类与判定: ZYYXH/T 157—2009[S]. 2009.
China Association of Chinese Medicine. Classification and determination of constitution in TCM: ZYYXH/T 157—2009[S]. 2009.
[124] ZHOU H, HU G, ZHANG X. Constitution identification of tongue image based on CNN[C]//Proceedings of the 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics, Beijing, Oct 13-15, 2018. Piscataway: IEEE, 2018: 1-5.
[125] 周浩, 胡广芹, 张新峰. 基于舌图像深度特征融合的中医体质分类方法研究[J]. 北京生物医学工程, 2020, 39(3): 221-226.
ZHOU H, HU G Q, ZHANG X F. Preliminary study on traditional Chinese medicine constitution classification method based on tongue image feature fusion[J]. Beijing Biomedical Engineering, 2020, 39(3): 221-226.
[126] 谢佳澄. 深度学习在中医舌象分类中的应用与实践[D]. 长春: 吉林大学, 2019.
XIE J C. Application and practice of deep learning in classification of tongue image in traditional Chinese medicine[D]. Changchun：Jilin University, 2019.
[127] LI H, WEN G, ZENG H. Natural tongue physique ident-ification using hybrid deep learning methods[J]. Multimedia Tools and Applications, 2019, 78(6): 6847-6868.
[128] LI T, WU C, MA Y. Multi-label constitution identification based on tongue image in traditional Chinese medicine[C]//Proceedings of the 2021 China Automation Congress, Beijing, Oct 22-24, 2021. Piscataway: IEEE, 2021: 1617-1622.
[129] JIANG T, HU X J, YAO X H, et al. Tongue image quality assessment based on a deep convolutional neural network [J]. BMC Medical Informatics and Decision Making, 2021, 21(1): 1-14.
[130] BARDOU D, ZHANG K, AHMAD S M. Lung sounds classification using convolutional neural networks[J]. Artificial Intelligence in Medicine, 2018, 88: 58-69.
[131] GANDOMKAR Z, BRENNAN P C, MELLO-THOMS C. MuDeRN: multi-category classification of breast histopat-hological image using deep residual networks[J]. Artificial Intelligence in Medicine, 2018, 88: 14-24.
[132] MA J, WEN G, WANG C, et al. Complexity perception classification method for tongue constitution recognition [J]. Artificial Intelligence in Medicine, 2019, 96: 123-133.
[133] JIANG Y, CHI Z. A CNN model for semantic person part segmentation with capacity optimization[J]. IEEE Transa-ctions on Image Processing, 2018, 28(5): 2465-2478.
[134] SANDLER M, HOWARD A, ZHU M, et al. MobileNetv2: inverted residuals and linear bottle-necks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Piscataway: IEEE, 2018: 4510-4520.
[135] HOWARD A, SANDLER M, CHU G, et al. Searching for MobileNetv3[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1314-1324.