Coarse-to-Fine Two-Stage Convolutional Neural Network Algorithm

doi:10.3778/j.issn.1673-9418.2006085

Abstract

Abstract:

In medicine, identifying the indirect immunofluorescence of human epithelial type 2 (HEp-2) cells plays a decisive role in the diagnosis of autoimmune diseases, which is limited by burden of human and material resources. Inspired by the outstanding performance of neural network in image classification tasks, a coarse-to-fine two-stage convolutional neural network (CTFTCNN) is proposed using clustering algorithm for classifying HEp-2 cells. In the proposed method, there are two types of classification tasks: coarse-grained classification and fine-grained classification. In coarse-grained classification, a clustering algorithm is first used to generate a coarse-grained dataset from the original dataset, and then a multi-scale convolutional neural network (MSCNN) is used to process the coarse-grained dataset. Next, fine-grained classification is performed under certain conditions. If each coarse class in coarse-grained dataset contains at least two fine classes, the coarse class will be subdivided further by using a VGG16 network. Finally, the two tasks handled by the coarse-grained and the fine-grained networks are combined together. For a coarse class that contains at least two fine classes, the features extracted from both the coarse-grained and fine-grained networks are merged to determine the final prediction result. Experiments are conducted on the real-world dataset to evaluate the proposed model. Experimental results show that CTFTCNN is promising compared with the state-of-the-art methods.

Key words: image classification, human epithelial type 2 (HEp-2) cell, convolutional neural network (CNN), coarse- to-fine scheme

摘要：

在医学上，人上皮2型（HEp-2）细胞的间接免疫荧光检测在自身免疫性疾病的诊断中起着决定性的作用，而自身免疫性疾病的诊断，往往受到人力物力的限制。鉴于神经网络在图像分类任务中的优异性能，提出了一种基于聚类算法的粗-细两阶段卷积神经网络算法（CTFTCNN），并应用到HEp-2细胞分类中。在所提出的方法中，有两种类型的分类任务：粗粒度分类和细粒度分类。粗粒度分类是指，采用聚类算法从原始数据集中生成一个粗粒度数据集，用多尺度卷积神经网络（MSCNN）去处理该粗粒度数据集。然后在一定条件下进行细粒度分类。在细粒度分类时，仅对在粗粒度分类中至少包含了两个细类的粗类进行处理，且采用VGG16网络对每个这样的粗类进行细分。最后集成粗粒度网络和细粒度网络的结果。具体地，对于至少包含了两个细类的粗类，将粗粒度和细粒度网络中提取的特征融合起来决定最终的预测结果。在真实数据集上进行实验以评估所提出的模型。实验结果表明：与目前最先进的方法相比，该模型具有良好的应用前景。

关键词: 图像分类, 人上皮2型（HEp-2）细胞, 卷积神经网络（CNN）, 粗到细策略

ZHANG Mengqian, ZHANG Li. Coarse-to-Fine Two-Stage Convolutional Neural Network Algorithm[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(8): 1501-1510.

张梦倩, 张莉. 粗-细两阶段卷积神经网络算法[J]. 计算机科学与探索, 2021, 15(8): 1501-1510.

References

[1] LECUN Y, BOTTOU L, BENGIO Y. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278 - 2324.
[2] KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25(2): 1106-1114.
[3] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of the 3rd International Conference on Learning Representations, Canada, May 7-9, 2015.
[4] SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Bosto, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 1-9.
[5] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[6] HUANG G, LIU Z, WEINBERGER K Q. Densely connected convolutional networks[J]. arXiv:1608.06993, 2016.
[7] ZHENG X, CAI D, HE X F, et al. Locality preserving clustering for image database[C]//Proceedings of the 12th ACM International Conference on Multimedia, New York, Oct 10-16, 2004. New York: ACM, 2004: 885-891.
[8] FERREIRA D S, RAMALHO G L B, DE MEDEIROS F N S, et al. Saliency-driven system with deep learning for cell image classification[C]//Proceedings of the 16th IEEE International Symposium on Biomedical Imaging, Venice, Apr 8-11, 2019. Piscataway: IEEE, 2019: 1284-1287.
[9] THEERA-UMPON N, DHOMPONGSA S. Morphological granulometric features of nucleus in automatic bone marrow white blood cell classification[J]. IEEE Transactions on Information Technology in Biomedicine, 2007, 11(3): 353-359.
[10] WANG R H, MCCANE B. Red blood cell classification through depth map and surface feature[C]//Proceedings of the 2008 International Symposium on Computer Science and Computational Technology, Shanghai, Dec 20-22, 2008. Washington: IEEE Computer Society, 2008: 339-342.
[11] GAO Z, ZHANG J, ZHOU L, et al. HEp-2 cell image classification with deep convolutional neural networks[J]. IEEE Journal of Biomedical and Health Informatics, 2017, 21(2): 416-428.
[12] NAVON D. Forest before trees: the precedence of global features in visual perception[J]. Cognitive Psychology, 1977, 9(3): 353-383.
[13] YANG G, HUANG T S. Human face detection in a complex background[J]. Pattern Recognition, 1994, 27(1): 53-63.
[14] TIAN X, GUO R, WU Q B, et al. Leucorrhea-wet-film recognition based on coarse-to-fine CNN-SVM[C]//Proceedings of the 2017 International Conference on Security, Pattern Analysis, and Cybernetics, Shenzhen, Dec 15-17, 2017. Piscataway: IEEE, 2017: 548-551.
[15] FU R, LI B, GAO Y, et al. CNN with coarse-to-fine layer for hierarchical classification[J]. IET Computer Vision, 2018, 12(6): 892-899.
[16] XIAO X, YUNNENG Y. Combination of multi-scale convolutional networks and SVM for SAR ATR[C]//Proceedings of the 2018 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference, Xi??an, May 25-27, 2018. Piscataway: IEEE, 2018: 66-69.
[17] TONTI S, CATALDO S D, Bottino A, et al. An automated approach to the segmentation of HEp-2 cells for the indirect immunofluorescence ANA test[J]. Computerized Medical Imaging and Graphics, 2015, 40: 62-69.
[18] LI Y X, SHEN L L. A deep residual inception network for HEp-2 cell classification[C]//LNCS 10553: Proceedings of the 2017 Deep Lepdfarning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Québec City, Sep 14, 2017. Berlin, Heidelberg: Springer, 2017: 12-20.
[19] HU C J, ZHOU R F, ZHANG S L, et al. Interpretation of the international consensus on standardized indirect immunofluorescence nomenclature of antinuclear antibody HEp-2 cell patterns and reporting ANA results[J]. Chinese Journal of Laboratory Medicine, 2016, 39(11): 804-810.
胡朝军, 周仁芳, 张蜀澜, 等. 抗核抗体HEp-2细胞间接免疫荧光模型及其结果报告方式国际共识解读[J]. 中华检验医学杂志, 2016, 39(11): 804-810.
[20] XU X, LIN F, NG C, et al. Automated classification for HEp-2 cells based on linear local distance coding framework[J]. EURASIP Journal Image and Video Processing, 2015(1): 13.
[21] WILIEM A, SANDERSON C, WONG Y, et al. Automatic classification of human epithelial type 2 cell indirect immunofluorescence images using cell pyramid matching[J]. Pattern Recognition, 2014, 47(7): 2315-2324.
[22] THEODORAKOPOULOS I, KASTANIOTIS D, ECONOMOU G, et al. HEp-2 cells classification via sparse representation of textural features fused into dissimilarity space[J]. Pattern Recognition, 2014, 47(7): 2367-2378.
[23] NOSAKA R, FUKUI K. HEp-2 cell classification using rotation invariant co-occurrence among local binary patterns[J]. Pattern Recognition, 2014, 47(7): 2428-2436.
[24] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[25] HAN X H, WANG J, XU G, et al. High-order statistics of microtexton for HEp-2 staining pattern classification[J]. IEEE Transactions on Biomedical Engineering, 2014, 61(8): 2223-2234.
[26] GUILLAUME T, JESúS A, FERNAND M. Advanced statistical matrices for texture characterization: application to cell classification[J]. IEEE Transactions on Biomedical Engineering, 2014, 61(3): 630-637.
[27] LIU J X, XU B L, SHEN L L, et al. HEp-2 cell classification based on a deep autoencoding-classification convolutional neural network[C]//Proceedings of the 14th IEEE International Symposium on Biomedical Imaging, Melbourne, Apr 18-21, 2017. Piscataway: IEEE, 2017: 1019-1023.
[28] SHEN L, JIA X, LI Y. Deep cross residual network for HEp-2 cell staining pattern classification[J]. Pattern Recognition, 2018, 82: 68-78.
[29] XIE H, HE Y J, LEI H J, et al. Deeply supervised residual network for HEp-2 cell classification[C]//Proceedings of the 24th International Conference on Pattern Recognition, Beijing, Aug 20-24, 2018. Washington: IEEE Computer Society, 2018: 699-703.
[30] YANG Y, XU D, NIE F, et al. Image clustering using local discriminant models and global integration[J]. IEEE Transactions on Image Processing, 2010, 19(10): 2761-2773.
[31] HE X F, CAI D, LIU H F, et al. Image clustering with tensor representation[C]//Proceedings of the 13th ACM International Conference on Multimedia, Singapore, Nov 6-11, 2005. New York: ACM, 2005: 132-140.
[32] TARIQ A, FOROOSH H. T-clustering: image clustering by tensor decomposition[C]//Proceedings of the 2015 IEEE International Conference on Image Processing, Quebec City, Sept 27-30, 2015. Piscataway: IEEE, 2015: 4803-4807.
[33] XIE J Y, HOU Q, CAO J W. Image clustering algorithms by deep convolutional autoencoders[J]. Journal of Frontiers of Computer Science and Technology, 2019, 13(4): 586-595.
谢娟英, 侯琦, 曹嘉文. 深度卷积自编码图像聚类算法[J]. 计算机科学与探索, 2019, 13(4): 586-595.
[34] WAN J, WU F, HE Y B, et al. Clustering algorithm for high-dimensional data under new dimensionality reduction criteria[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(1): 96-107.
万静, 吴凡, 何云斌, 等. 新的降维标准下的高维数据聚类算法[J]. 计算机科学与探索, 2020, 14(1): 96-107.
[35] HONG M, JIA C Y, WANG X Y. Research on initialization of K-means type multi-view clustering[J]. Journal of Frontiers of Computer Science and Technology, 2019, 13(4): 574-585.
洪敏, 贾彩燕, 王晓阳. K-means型多视图聚类中的初始化问题研究[J]. 计算机科学与探索, 2019, 13(4): 574-585.
[36] SERMANET P, LECUN Y. Traffic sign recognition with multi-scale convolutional networks[C]//Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, Jul 31-Aug 5, 2011. Piscataway: IEEE, 2011: 2809-2813.
[37] HOBSON P, LOVELL B C, Percannella G, et al. Classifying anti-nuclear antibodies HEp-2 images: a benchmarking platform[C]//Proceedings of the 22nd International Conference on Pattern Recognition, Stockholm, Aug 24-28, 2014. Washington: IEEE Computer Society, 2014: 3233-3238.
[38] WIIK A S, H?IER-MADSEN M, FORSLID J, et al. Antinuclear antibodies: a contemporary nomenclature using HEp-2 cells[J]. Journal of Autoimmunity, 2010, 35(3): 276-290.
[39] HOBSON P, LOVELL B C, PERCANNELLA G, et al. HEp-2 staining pattern recognition at cell and specimen levels: datasets, algorithms and results[J]. Pattern Recognition Letters, 2016, 82: 12-22.
[40] QI X, ZHAO G, CHEN J, et al. Exploring illumination robust descriptors for human epithelial type 2 cell classification[J]. Pattern Recognition, 2016, 60: 420-429.