Development and Application of Convolutional Neural Network Model

doi:10.3778/j.issn.1673-9418.2008016

Abstract

Abstract:

Deep learning is the latest trend of machine learning and artificial intelligence research. As a new field of rapid development in the past ten years, more and more researchers pay attention to it. Convolutional neural network (CNN) model is one of the most important classical structures in deep learning model, and its performance has been gradually improved in recent years. Since it can automatically learn the feature representation of sample data, convolutional neural network has been widely used in image classification, object detection, semantic segmentation and natural language processing. This paper first analyzes the model structure of typical convolutional neural network model, which increases the depth and width of the network in order to improve its performance, analyzes the network structure that uses attention mechanism to further improve the performance of the model, and then summarizes and analyzes the current special model structure. Finally, it summarizes and discusses the application of convolutional neural network in related fields, and the future research direction is prospected.

Key words: convolutional neural network (CNN) model, feature extraction, computer vision, natural language processing

摘要：

深度学习是机器学习和人工智能研究的最新趋势，作为一个十余年来快速发展的崭新领域，越来越受到研究者的关注。卷积神经网络（CNN）模型是深度学习模型中最重要的一种经典结构，其性能在近年来深度学习任务上逐步提高。由于可以自动学习样本数据的特征表示，卷积神经网络已经广泛应用于图像分类、目标检测、语义分割以及自然语言处理等领域。首先分析了典型卷积神经网络模型为提高其性能增加网络深度以及宽度的模型结构，分析了采用注意力机制进一步提升模型性能的网络结构，然后归纳分析了目前的特殊模型结构，最后总结并讨论了卷积神经网络在相关领域的应用，并对未来的研究方向进行展望。

关键词: 卷积神经网络（CNN）模型, 特征提取, 计算机视觉, 自然语言处理

YAN Chunman, WANG Cheng. Development and Application of Convolutional Neural Network Model[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(1): 27-46.

严春满, 王铖. 卷积神经网络模型发展及应用[J]. 计算机科学与探索, 2021, 15(1): 27-46.

References

[1] SUN B, JU Q Q, SANG Q B. Image dehazing algorithm based on FC-DenseNet and WGAN[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(8): 1380-1388.
孙斌, 雎青青, 桑庆兵. 结合FC-DenseNet和WGAN的图像去雾算法[J]. 计算机科学与探索, 2020, 14(8): 1380-1388.
[2] SAMPEDRO C, RODRIGUEZ-VAZQUEZ J, RODRIGUEZ-RAMOS A, et al. Deep learning-based system for automatic recognition and diagnosis of electrical insulator strings[J]. IEEE Access, 2019, 7: 101283-101308.
[3] HARIKRISHNAN J, SUDARSAN A, SADASHIV A, et al. Vision-face recognition attendance monitoring system for surveillance using deep learning technology and computer vision[C]//Proceedings of the 2019 International Conference on Vision Towards Emerging Trends in Communication and Networking, Vellore, Mar 30-31, 2019. Piscataway: IEEE, 2019: 1-5.
[4] LIN K Z, BAI J X, LI H T, et al. Facial expression recognition with small samples fused with different models under deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(3): 482-492.
林克正, 白婧轩, 李昊天, 等. 深度学习下融合不同模型的小样本表情识别[J]. 计算机科学与探索, 2020, 14(3): 482-492.
[5] WANG Z Y, YUAN C, LI J C. Instance segmentation with separable convolutions and multi-level features[J]. Journal of Software, 2019, 30(4): 954-961.
王子愉, 袁春, 黎健成. 利用可分离卷积和多级特征的实例分割[J]. 软件学报, 2019, 30(4): 954-961.
[6] XIAO L, CHEN B L, HUANG X, et al. Multi-label text classification method based on label semantic information[J]. Journal of Software, 2020, 31(4): 1079-1089.
肖琳, 陈博理, 黄鑫, 等. 基于标签语义注意力的多标签文本分类[J]. 软件学报, 2020, 31(4): 1079-1089.
[7] JAF S, CALDER C. Deep learning for natural language par-sing[J]. IEEE Access, 2019, 7: 131363-131373.
[8] ROBERTS L G. Machine perception of three-dimensional solids[D]. Cambridge: Massachusetts Institute of Technology, 1963.
[9] PELLEGRINO F A, VANZELLA W, TORRE V. Edge detection revisited[J]. IEEE Transactions on Systems, Man, and Cyber-netics, Part B, 2004, 34(3): 1500-1518.
[10] MARR D, HILDRETH E. Theory of edge detection[J]. Pro-ceedings of the Royal Society B: Biological Sciences, 1980, 207(1167): 187-217.
[11] CANNY J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intell-igence, 1986(6): 679-698.
[12] HARRIS C G, STEPHENS M. A combined corner and edge detector[C]//Proceedings of the Alvey Vision Conference, Manchester, Sep, 1988: 1-6.
[13] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Sep 20-27, 1999. Washington: IEEE Computer Society, 1999: 1150-1157.
[14] ROSTEN E, PORTER R, DRUMMOND T. Faster and better: a machine learning approach to corner detection[J]. IEEE Transactions on Pattern Analysis and Machine Inte-lligence, 2008, 32(1): 105-119.
[15] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[16] COVER T, HART P. Nearest neighbor pattern classification[J]. IEEE Transactions on Information Theory, 1967, 13(1): 21-27.
[17] ZHAO Q, PRINCIPE J C. Support vector machines for SAR automatic target recognition[J]. IEEE Transactions on Aerospace and Electronic Systems, 2001, 37(2): 643-654.
[18] THIAGARAJAN J J, RAMAMURTHY K N, KNEE P, et al. Sparse representations for automatic target classification in SAR images[C]//Proceedings of the 2010 4th International Symposium on Communications, Control and Signal Pro-cessing, Limassol, Mar 3-5, 2010. Piscataway: IEEE, 2010: 1-4.
[19] MCCULLOCH W S, PITTS W. A logical calculus of the ideas immanent in nervous activity[J]. The Bulletin of Math-ematical Biophysics, 1943, 5(4): 115-133.
[20] ROSENBLATT F. The perceptron: a probabilistic model for information storage and organization in the brain[J]. Psych-ological Review, 1958, 65(6): 386-408.
[21] MYCIELSKI J. Book review: perceptrons, an introduction to computational geometry[J]. Bulletin of The American Mathematical Society, 1972, 78(1): 12-16.
[22] RUMELHART D E, Hinton G E, Williams R J. Learning representations by back propagating errors[J]. Nature, 1986, 323(6088): 533-536.
[23] HOCHREITER S. Untersuchungen zu dynamischen neuronalen Netzen[R]. Linz: Johannes Kepler University Linz, 1991: 1-65.
[24] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief net[J]. Neural Computation, 2006, 18(7): 1527-1554．
[25] LECUN Y, BOTTOU L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[26] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 26th Annual Conference on Neural Infor-mation Processing Systems, Lake Tahoe, Dec 3-6, 2012. Red Hook: Curran Associates, 2012: 1106-1114.
[27] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//LNCS 8689: Proceedings of the 13th European Conference on Computer Vision, Zurich, Sep 6-12, 2014. Berlin, Heidelberg: Springer, 2014: 818-833.
[28] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large scale image recognition[J]. arXiv:1409. 1556, 2014.
[29] HE K M, ZHANG X Y, REN S Q, et al. Delving deep into rectifiers: surpassing human-level performance on imagenet classification[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 1026-1034.
[30] TAN M X, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[C]//Proceedings of the 36th International Conference on Machine Learning, Long Beach, Jun 9-15, 2019: 6105-6114.
[31] LIN M, CHEN Q, YAN S C. Networkin network[C]//Proceedings of the 2nd International Conference on Learning Representations, Banff, Apr 14-16, 2014: 1-10.
[32] SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 1-9.
[33] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 2818-2826.
[34] HOCHREITER S, SCHMIDHUBER J. Long short-term me-mory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[35] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recog-nition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Com-puter Society, 2016: 770-778.
[36] XIE S N, GIRSHICK R B, DOLLáR P, et al. Aggregated residual transformations for deep neural networks[C]//Proc-eedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Wash-ington: IEEE Computer Society, 2017: 5987-5995.
[37] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 2261-2269.
[38] Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, Incep-tion-ResNet and the impact of residual connections on learning[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, Feb 4-9, 2017. Menlo Park:AAAI, 2017: 4278-4284.
[39] YAMADA Y, IWAMURA M, KISE K. Deep pyramidal residual networks with separated stochastic depth[J]. arXiv:1612.01230, 2016.
[40] ZAGORUYKO S, KOMODAKIS N. Wide residual networks[J]. arXiv:1605.07146, 2016.
[41] ZHANG X, LI Z, CHANGE LOY C, et al. PolyNet: a pursuit of structural diversity in very deep networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 3900-3908.
[42] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 7132-7141.
[43] LI X, WANG W H, HU X L, et al. Selective kernel networks[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 510-519.
[44] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//LNCS 11211: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Berlin, Heidelberg: Springer, 2018: 3-19.
[45] IANDOLA F N, HAN S, MOSKEWICZ M W, et al. Squ-eezeNet: AlexNet-level accuracy with 50x fewer parameters and<0.5 MB model size[J]. arXiv:1602.07360, 2016.
[46] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, Mon-treal, Dec 8-13, 2014. Red Hook: Curran Associates, 2014: 2672-2680.
[47] ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks[C]//Proc-eedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 4353-4361.
[48] HAN K, WANG Y H, TIAN Q, et al. GhostNet: more features from cheap operations[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 1577-1586.
[49] GIRSHICK R B, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Jun 23-28, 2014. Washington: IEEE Computer Society, 2014: 580-587.
[50] GIRSHICK R B. Fast R-CNN[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 1440-1448．
[51] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelli-gence, 2017, 39(6): 1137-1149.
[52] DAI J F, LI Y, HE K M, et al. R-FCN: object detection via region-based fully convolutional networks[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates, 2016: 379-387.
[53] GIDARIS S, KOMODAKIS N. Object detection via a multi- region and semantic segmentation-aware CNN model[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 1134-1142.
[54] SERMANET P, EIGEN D, ZHANG X, et al. OverFeat: integrated recognition, localization and detection using con-volutional networks[J]. arXiv:1312.6229, 2013.
[55] REDMON J, DIVVALA S K, GIRSHICK R B, et al. You only look once: unified, real-time object detection[C]//Pro-ceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Wash-ington: IEEE Computer Society, 2016: 779-788.
[56] LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//LNCS 9905: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Berlin, Heidelberg: Springer, 2016: 21-37.
[57] REDMON J, FARHADI A. YOLO9000: better, faster, str-onger[C]//Proceedings of the 2017 IEEE Conference on Com-puter Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 7263-7271.
[58] FU C Y, LIU W, RANGA A, et al. DSSD: deconvolutional single shot detector[J]. arXiv:1701.06659, 2017.
[59] REDMON J, FARHADI A. YOLOv3: an incremental impro-vement[J]. arXiv:1804.02767, 2018.
[60] SHEN Z Q, LIU Z, LI J G, et al. DSOD: learning deeply supervised object detectors from scratch[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 1937-1945.
[61] LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 8759-8768.
[62] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[J]. arXiv:2004.10934, 2020.
[63] LAW H, HENG J. Cornernet: detecting objects as paired keypoints[C]//LNCS 11218: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Berlin, Heidelberg: Springer, 2018: 765-781.
[64] ZHOU X Y, ZHOU J C, KR?HENBüHL P. Bottom up object detection by grouping extreme and center points[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 850-859.
[65] DUAN K, BAI S, XIE L X, et al. Centernet: keypoint triplets for object detection[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 6569-6578.
[66] ZHU C C, HE Y H, SAVVIDES M. Feature selective anchor-free module for single-shot object detection[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway:IEEE, 2019: 840-849.
[67] TIAN Z, SHEN C H, CHEN H, et al. FCOS: fully convo-lutional one-stage object detection[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 9626-9635.
[68] VAILLANT R, MONROCQ C, LE CUN Y. Original approach for the localisation of objects in images[J]. Vision, Image and Signal Processing, 1994, 141(4): 245-250.
[69] SERMANET P, KAVUKCUOGLU K, CHINTALA S, et al. Pedestrian detection with unsupervised multi-stage feature learning[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, Jun 23-28, 2013. Piscataway: IEEE, 2013: 3626-3633.
[70] HU C, BAI X, QI L, et al. Vehicle color recognition with spatial pyramid deep learning[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(5): 2925-2934.
[71] SERMANET P, LECUN Y. Traffic sign recognition with multi-scale convolutional networks[C]//Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, Jul 31-Aug 5, 2011. Piscataway: IEEE, 2011: 2809-2813.
[72] WANG L, WONG A. COVID-Net: a tailored deep con-volutional neural network design for detection of COVID-19 cases from chest X-Ray images[J]. arXiv:2003.09871, 2020.
[73] HUANG Z J, SANG Q B. Ship detection based on improved R-FCN[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(6): 1045-1053.
黄致君, 桑庆兵. 改进R-FCN的船舶识别方法[J]. 计算机科学与探索, 2020, 14(6): 1045-1053.
[74] LUO X Q, PAN S L. Improved YOLOV3 fire detection method[J]. Computer Engineering and Applications, 2020, 56(17): 187-196.
罗小权, 潘善亮. 改进YOLOV3的火灾检测方法[J]. 计算机工程与应用, 2020, 56(17): 187-196.
[75] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Com-puter Society, 2015: 3431-3440.
[76] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[77] CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//LNCS 11211: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Berlin, Heidelberg: Springer, 2018: 833-851.
[78] RONNEBERGER O, FISCHER P, BROX T. U-net: con-volutional networks for biomedical image segmentation[C]//LNCS 9351: Proceedings of the 18th International Con-ference on Medical Image Computing and Computer-Assisted Intervention, Munich, Oct 5-9, 2015. Berlin, Heidelberg: Springer, 2015: 234-241.
[79] MILLETARI F, NAVAB N, AHMADI S A. V-net: fully convolutional neural networks for volumetric medical image segmentation[C]//Proceedings of the 4th International Con-ference on 3D Vision, Stanford, Oct 25-28, 2016. Washington: IEEE Computer Society, 2016: 565-571.
[80] LIU W, RABINOVICH A, BERG A C. Parsenet: looking wider to see better[J]. arXiv:1506.04579, 2015.
[81] LIN G S, MILAN A, SHEN C H, et al. RefineNet: multi-path refinement networks for high-resolution semantic segmentation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 5168-5177.
[82] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 834-848.
[83] ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 6230-6239.
[84] YUAN Y, WANG J. OCNet: object context network for scene parsing[J]. arXiv:1809.00916, 2018.
[85] Al-ARIFS M M R, KNAPP K, SLABAUGH G G. SPNet: shape prediction using a fully convolutional neural network[C]//LNCS 11070: Proceedings of the 21st International Conference on Medical Image Computing and Computer Assisted Intervention, Granada, Sep 16-20, 2018. Berlin, Heidelberg: Springer, 2018: 430-439.
[86] YU C Q, WANG J B, GAO C X, et al. Context prior for scene segmentation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 12416-12425.
[87] PENG C, ZHANG X Y, YU G, et al. Large kernel matters—improve semantic segmentation by global convolutional network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 1743-1751.
[88] LI H, XIONG P, AN J, et al. Pyramid attention network for semantic segmentation[J]. arXiv:1805.10180, 2018.
[89] YU C Q, WANG J B, PENG C, et al. Learning a discrim-inative feature network for semantic segmentation[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 1857-1866.
[90] FU J, LIU J, TIAN H J, et al. Dual attention network for scene segmentation[C]//Proceedings of the 2019 IEEE Con-ference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 3146-3154.
[91] YANG L, HAN Y Z, CHEN X, et al. Resolution adaptive networks for efficient inference[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 2366-2375.
[92] WANG Y D, ZHANG J, KAN M, et al. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 12272-12281.
[93] CHOI S, KIM J T, CHOO J. Cars can??t fly up in the sky: improving urban-scene segmentation via height-driven atten-tion networks[C]//Proceedings of the 2020 IEEE/CVF Con-ference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 9370-9380.
[94] LEE S, KIM J, SHIN YOON J, et al. VPGNet: vanishing point guided network for lane and road marking detection and recognition[C]//Proceedings of the 2017 IEEE Intern-ational Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 1965-1973.
[95] DIAKOGIANNIS F I, WALDNER F, CACCETTA P, et al. ResUNet-a: a deep learning framework for semantic seg-mentation of remotely sensed data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 162: 94-114.
[96] TRAN T, KWON O H, KWON K R, et al. Blood cell images segmentation using deep learning semantic segmentation [C]//Proceedings of the 2018 IEEE International Con-ference on Electronics and Communication Engineering, Xi??an, Dec 10-12, 2018. Piscataway: IEEE, 2018: 13-16.
[97] EDUPUGANTI V G, CHAWLA A, KALE A. Automatic optic disk and cup segmentation of fundus images using deep learning[C]//Proceedings of the 2018 IEEE International Conference on Image Processing, Athens, Oct 7-10, 2018. Piscataway: IEEE, 2018: 2227-2231.
[98] LIU C, XIAO Z Y, WU X X. Application of three-dim-ensional convolution network in brain hippocampus segm-entation[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(3): 493-501.
刘辰, 肖志勇, 吴鑫鑫. 三维卷积网络在脑海马体分割中的应用[J]. 计算机科学与探索, 2020, 14(3): 493-501.
[99] WANG Y G, XI Y Y, PAN X Y. Method for intestinal Polyp segmentation by improving DeepLabv3+ network[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(7): 1243-1250.
王亚刚, 郗怡媛, 潘晓英. 改进DeepLabv3+网络的肠道息肉分割方法[J]. 计算机科学与探索, 2020, 14(7): 1243-1250.
[100] WANG Z, SHI Y, LI Y B. Segmentation of corn leaf diseases based on improved fully convolutional neural network[J]. Computer Engineering and Applications, 2019, 55(22): 127-132.
王振, 师韵, 李玉彬. 基于改进全卷积神经网络的玉米叶片病斑分割[J]. 计算机工程与应用, 2019, 55(22): 127-132.
[101] COLLOBERT R, WESTON J. A unified architecture for natural language processing: deep neural networks with multitask learning[C]//Proceedings of the 25th International Conference on Machine Learning, Helsinki, Jun 5-9, 2008. New York: ACM, 2008: 160-167.
[102] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12: 2493-2537.
[103] KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stroudsburg: ACL, 2014: 1746-1751.
[104] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[J]. arXiv:1404.2188, 2014.
[105] JOHNSON R, ZHANG T. Deep pyramid convolutional neural networks for text categorization[C]//Proceedings of the 55th Annual Meeting of the Association for Com-putational Linguistics, Vancouver, Jul 30-Aug 4, 2017. Stroudsburg: ACL, 2017: 562-570.
[106] WANG R S, LI Z, CAO J, et al. Convolutional recurrent neural networks for text classification[C]//Proceedings of the 2019 International Joint Conference on Neural Networks, Budapest, Jul 14-19, 2019. Piscataway: IEEE, 2019: 1-6.
[107] KALCHBRENNER N, ESPEHOLT L, SIMONYAN K, et al. Neural machine translation in linear time[J]. arXiv:1610.10099, 2016.
[108] YU L T, ZHANG W N, WANG J, et al. SeqGAN: sequence generative adversarial nets with policy gradient[C]//Procee-dings of the 31st AAAI Conference on Artificial Inte-lligence, San Francisco, Feb 4-9, 2017. Menlo Park: AAAI, 2017: 2852-2858.
[109] YAN D, HE J, LIU H Y, et al. Considering grade infor-mation for music comment text automatic generation[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(8): 1389-1396.
严丹, 何军, 刘红岩, 等. 考虑评级信息的音乐评论文本自动生成[J]. 计算机科学与探索, 2020, 14(8): 1389-1396.
[110] TU W B, YUAN Z M, YU K. Convolutional neural networks without pooling layer for Chinese word segmentation[J]. Computer Engineering and Applications, 2020, 56(2): 120-126.
涂文博, 袁贞明, 俞凯. 无池化层卷积神经网络的中文分词方法[J]. 计算机工程与应用, 2020, 56(2): 120-126.