Survey of Point Cloud Semantic Segmentation Based on Deep Learning

doi:10.3778/j.issn.1673-9418.2006025

Abstract

Abstract:

In recent years, the popularity of depth sensors and 3D laserscanners has led to a rapid development of 3D point clouds processing methods. Semantic segmentation of point cloud, as a key step in understanding 3D scenes, has attracted extensive attention of researchers. With the rapid development of deep learning and its widespread applications in 3D semantic segmentation, the quality of point cloud semantic segmentation has been significantly improved. This paper mainly reviews the deep learning-based point cloud semantic segmentation methods and their research status. This paper categories these deep learning-based methods for point clouds into two groups: indirect and direct semantic segmentation methods. In terms of the characteristics of the algorithm, each of groups is further subdivided. The representative algorithms are analyzed and introduced. This paper summarizes the theories, principles, advantages and disadvantages of each type of method, and systematically explains the contribution of deep learning to the field of semantic segmentation. Moreover, the current mainstream datasets and remote sensing datasets are summarized and the experimental results of some algorithms are compared. Finally, the future development direction of semantic segmentation technology is prospected.

Key words: deep learning, semantic segmentation, point cloud, computer vision

摘要：

近年来，深度传感器和三维激光扫描仪的普及推动了三维点云处理方法的快速发展。点云语义分割作为理解三维场景的关键步骤，受到了研究者的广泛关注。随着深度学习的迅速发展并广泛应用到三维语义分割领域，点云语义分割效果得到了显著提升。主要对基于深度学习的点云语义分割方法和研究现状进行了详细的综述。将基于深度学习的点云语义分割方法分为间接语义分割方法和直接语义分割方法，根据各方法的研究内容进一步细分，对每类方法中代表性算法进行分析介绍，总结每类方法的基本思想和优缺点，并系统地阐述了深度学习对语义分割领域的贡献。然后，归纳了当前主流的公共数据集和遥感数据集，并在此基础上对比主流点云语义分割方法的实验结果。最后，对语义分割技术未来的发展方向进行了展望。

关键词: 深度学习, 语义分割, 点云, 计算机视觉

JING Zhuangwei, GUAN Haiyan, ZANG Yufu, NI Huan, LI Dilong, YU Yongtao. Survey of Point Cloud Semantic Segmentation Based on Deep Learning[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(1): 1-26.

景庄伟, 管海燕, 臧玉府, 倪欢, 李迪龙, 于永涛. 基于深度学习的点云语义分割研究综述[J]. 计算机科学与探索, 2021, 15(1): 1-26.

References

[1] ZHANG J, LIN X, NING X. SVM-based classification of segmented airborne LiDAR point clouds in urban areas[J]. Remote Sensing, 2013, 5(8): 3749-3775.
[2] SUN J, LAI Z L. Airbone LiDAR feature selection for urban classification using random forests[J]. Geomatics and Information Science of Wuhan University, 2014, 39(11): 1310-1313.
孙杰, 赖祖龙. 利用随机森林的城区机载 LiDAR 数据特征选择与分类[J]. 武汉大学学报(信息科学版), 2014, 39(11): 1310-1313.
[3] ZHUANG Y, LIU Y S, HE G J, et al. Contextual classification of 3D laser points with conditional random fields in urban environments[C]//Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems, Hamburg, Sep 28-Oct 2, 2015. Piscataway: IEEE, 2015: 3908-3913.
[4] LU Y, RASMUSSEN C. Simplified Markov random fields for efficient semantic labeling of 3D point clouds[C]//Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Oct 7-12, 2012. Piscataway: IEEE, 2012: 2690-2697.
[5] QI C R, SU H, MO K, et al. Pointnet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 652-660.
[6] GRIFFITHS D, BOEHM J. A review on deep learning techniques for 3D sensed data classification[J]. Remote Sensing, 2019, 11(12): 1499.
[7] XIE Y, TIAN J, ZHU X X. A review of point cloud semantic segmentation[J]. arXiv:1908.08854, 2019.
[8] ZHANG J Y, ZHAO X L, CHEN Z. Review of semantic segmentation of point cloud based on deep learning[J]. Laser & Optoelectronics Progress, 2020, 57(4): 20-38.
张佳颖, 赵晓丽, 陈正. 基于深度学习的点云语义分割综述[J]. 激光与光电子学进展, 2020, 57(4): 20-38.
[9] GUO Y, WANG H, HU Q, et al. Deep learning for 3D point clouds: a survey[J]. arXiv:1912.12033, 2019.
[10] JIANG M, WU Y, ZHAO T, et al. Pointsift: a sift-like network module for 3D point cloud semantic segmentation[J]. arXiv:1807.00652, 2018.
[11] LI J X, CHEN B M, LEE G H. SO-net: self-organizing network for point cloud analysis[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 9397-9406.
[12] LIANG Z D, YANG M, DENG L Y, et al. Hierarchical depthwise graph convolutional neural network for 3D semantic segmentation of point clouds[C]//Proceedings of the 2019 International Conference on Robotics and Automation, Montreal, May 20-24, 2019. Piscataway: IEEE, 2019: 8152-8158.
[13] HUANG Q G, WANG W Y, NEUMANN U. Recurrent slice networks for 3D segmentation of point clouds[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 2626-2635.
[14] LI Y Y, BU R, SUN M C, et al. PointCNN: convolution on x-transformed points[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, Montréal, Dec 3-8, 2018: 828-838.
[15] ZHANG Z Y, HUA B S, YEUNG S K. Shellnet: efficient point cloud convolutional neural networks using concentric shells statistics[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1607-1616.
[16] ZHANG J Y, ZHAO X L, CHEN Z, et al. A review of deep learning-based semantic segmentation for point cloud[J]. IEEE Access, 2019, 7: 179118-179133.
[17] QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 5099-5108.
[18] HU Q Y, YANG B, XIE L H, et al. RandLA-Net: efficient semantic segmentation of large-scale point clouds[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 11105-11114.
[19] WANG L, HUANG Y C, HOU Y L, et al. Graph attention convolution for point cloud semantic segmentation[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 10296-10305.
[20] YE X Q, LI J M, HUANG H X, et al. 3D recurrent neural networks with context fusion for point cloud semantic segmentation[C]//LNCS 11211: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Berlin, Heidelberg: Springer, 2018: 415-430.
[21] THOMAS H, QI C R, DESCHAUD J E, et al. Kpconv: flexible and deformable convolution for point clouds[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 6410-6419.
[22] MAO J G, WANG X G, LI H S. Interpolated convolutional networks for 3D point cloud understanding[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1578-1587.
[23] WU W, QI Z, FUXIN L. Pointconv: deep convolutional networks on 3D point clouds[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 9621-9630.
[24] WANG Y, SUN Y, LIU Z, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 1-12.
[25] TE G, HU W, ZHENG A, et al. RGCNN: regularized graph CNN for point cloud segmentation[C]//Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Oct 22-26, 2018. New York: ACM, 2018: 746-754.
[26] LIU J X, NI B B, LI C Y, et al. Dynamic points agglomeration for hierarchical point sets learning[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 7545-7554.
[27] BOULCH A. ConvPoint: continuous convolutions for point cloud processing[J]. Computers & Graphics, 2020, 88: 24-34.
[28] ENGELMANN F, KONTOGIANNI T, HERMANS A, et al. Exploring spatial context for 3D semantic segmentation of point clouds[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 716-724.
[29] WANG S L, SUO S, MA W C, et al. Deep parametric continuous convolutional neural networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 2589-2597.
[30] XU Y F, FAN T Q, XU M Y, et al. SpiderCNN: deep learning on point sets with parameterized convolutional filters[C]//LNCS 11212: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Berlin, Heidelberg: Springer, 2018: 90-105.
[31] SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 945-953.
[32] QI C R, SU H, NIE?NER M, et al. Volumetric and multi-view CNNs for object classification on 3D data[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 5648-5656.
[33] FENG Y F, ZHANG Z Z, ZHAO X B, et al. GVCNN: group-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 264-272.
[34] ZENG A, YU K T, SONG S R, et al. Multi-view self- supervised deep learning for 6D pose estimation in the Amazon picking challenge[C]//Proceedings of the 2017 IEEE International Conference on Robotics and Automation, Singapore, May 29-Jun 3, 2017. Piscataway: IEEE, 2017: 1386-1383.
[35] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Image-net classification with deep convolutional neural networks[C]//Proceedings of the 26th Annual Conference on Neural Information Processing Systems 2012, Lake Tahoe, Dec 3-6, 2012. Red Hook: Curran Associates, 2012: 1106-1114.
[36] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409. 1556, 2014.
[37] MA L N, STüCKLER J, KERL C, et al. Multi-view deep learning for consistent semantic mapping with RGB-D cameras[C]//Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vancouver, Sep 24-28, 2017. Piscataway: IEEE, 2017: 598-605.
[38] BOULCH A, GUERRY J, LE SAUX B, et al. SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks[J]. Computers & Graphics, 2018, 71: 189-198.
[39] GUERRY J, BOULCH A, LE SAUX B, et al. SnapNet-R: consistent 3D multi-view semantic labeling for robotics[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 669-678.
[40] WU B C, WAN A, YUE X Y, et al. SqueezeSeg: convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3DLiDAR point cloud[C]//Proceedings of the 2018 IEEE International Conference on Robotics and Automation, Brisbane, May 21-25, 2018. Piscataway: IEEE, 2018: 1887-1893.
[41] WU B C, ZHOU X Y, ZHAO S C, et al. SqueezeSegv2: improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud[C]//Proceedings of the 2019 International Conference on Robotics and Automation, Montreal, May 20-24, 2019. Piscataway: IEEE, 2019: 4376-4382.
[42] MATURANA D, SCHERER S A. VoxNet: a 3D convolutional neural network for real-time object recognition[C]//Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems, Hamburg, Sep 28-Oct 2, 2015. Piscataway: IEEE, 2015: 922-928.
[43] WU Z R, SONG S R, KHOSLA A, et al. 3DshapeNets: a deep representation for volumetric shapes[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 1912-1920.
[44] DAI A, CHANG A X, SAVVA M, et al. ScanNet: richly-annotated 3D reconstructions of indoor scenes[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 2432-2443.
[45] ÇIÇEK Ö, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]//LNCS 9901: Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, Athens, Oct 17-21, 2016. Berlin, Heidelberg: Springer, 2016: 424-432.
[46] LI Y Y, PIRK S, SU H, et al. FPNN: field probing neural networks for 3D data[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2016, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates, 2016: 307-315.
[47] TCHAPMI L P, CHOY C B, ARMENI I, et al. SEGCloud: semantic segmentation of 3D point clouds[C]//Proceedings of the 2017 International Conference on 3D Vision, Qingdao, Oct 10-12, 2017: 537-547.
[48] RIEGLER G, OSMAN ULUSOY A O, GEIGER A. OctNet: learning deep 3D representations at high resolutions[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 6620-6629.
[49] XU Y, HOEGNER L, TUTTAS S, et al. Voxel- and graph-based point cloud segmentation of 3D scenes using perceptual grouping laws[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, 5: 43-50.
[50] KLOKOV R, LEMPITSKY V S. Escape from cells: deep Kd-networks for the recognition of 3D point cloud models[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 863-872.
[51] ZENG W, GEVERS T. 3DContextNet: K-d tree guided hierarchical learning of point clouds using local and global contextual cues[C]//LNCS 11131: Proceedings of the ECCV 2018 Workshops on Computer Vision, Munich, Sep 8-14, 2018. Berlin, Heidelberg: Springer, 2018: 314-330.
[52] MENG H Y, GAO L, LAI Y K, et al. VV-Net: voxel VAE net with group convolutions for point cloud segmentation[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 8499-8507.
[53] WANG L, HUANG Y, SHAN J, et al. MSNet: multi-scale convolutional network for point cloud classification[J]. Remote Sensing, 2018, 10(4): 612.
[54] ZHAO H S, JIANG L, FU C W, et al. PointWeb: enhancing local neighborhood features for point cloud processing[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 5565-5573.
[55] XIE S N, LIU S N, CHEN Z Y, et al. Attentional shapecontextnet for point cloud recognition[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 4606-4615.
[56] WANG C, SAMARI B, SIDDIQI K. Local spectral graph convolution for point set feature learning[C]//LNCS 11208: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Berlin, Heidelberg: Springer, 2018: 56-71.
[57] SIMONOVSKY M, KOMODAKIS N. Dynamic edge-conditioned filters in convolutional neural networks on graphs[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 29-38.
[58] LU Q, CHEN C, XIE W, et al. PointNGCNN: deep convolutional networks on 3D point clouds with neighborhood graph filters[J]. Computers & Graphics, 2020, 86: 42-51.
[59] LANDRIEU L, SIMONOVSKY M. Large-scale point cloud semantic segmentation with superpoint graphs[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 4558-4567.
[60] HUA B S, TRAN M K, YEUNG S K. Pointwise convolutional neural networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 984-993.
[61] KOMARICHEV A, ZHONG Z, HUA J. A-CNN: annularly convolutional neural networks on point clouds[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 7421-7430.
[62] HIMMELSBACH M, HUNDELSHAUSEN F V, WüNS-CHEH J. Fast segmentation of 3D point clouds for ground vehicles[C]//Proceedings of the 2010 IEEE Intelligent Vehicles Symposium, La Jolla, Jun 21-24, 2010. Piscataway: IEEE, 2010: 560-565.
[63] LIU F Y, LI S P, ZHANG L Q, et al. 3DCNN-DQN-RNN: a deep reinforcement learning framework for semantic parsing of large-scale 3D point clouds[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 5678-5687.
[64] YANG J C, ZHANG Q, NI B B, et al. Modeling point clouds with self-attention and gumbel subset sampling[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 3323-3332.
[65] ZHAO C X, ZHOU W H, LU L, et al. Pooling scores of neighboring points for improved 3D point cloud segmentation[C]//Proceedings of the 2019 IEEE International Conference on Image Processing, Taipei, China, Sep 22-25, 2019. Piscataway: IEEE, 2019: 1475-1479.
[66] CHEN C, FRAGONARA L Z, TSOURDOS A. GAPNet: graph attention based point neural network for exploiting local feature of point cloud[J]. arXiv:1905.08705, 2019.
[67] WANG X L, LIU S, SHEN X Y, et al. Associatively segmenting instances and semantics in point clouds[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 4096-4105.
[68] PHAM Q H, NGUYEN T, HUA B S, et al. JSIS3D: joint semantic-instance segmentation of 3D point clouds with multi-task pointwise networks and multi-value conditional random fields[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 8827-8836.
[69] LAI K, BO L, REN X F, et al. A large-scale hierarchical multi-view RGB-D object dataset[C]//Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, May 9-13, 2011. Piscataway: IEEE, 2011: 1817-1824.
[70] SILBERMAN N, HOIEM D, KOHLI P, et al. Indoor segmentation and support inference from RGBD images[C]//LNCS 7576: Proceedings of the 12th European Conference on Computer Vision, Florence, Oct 7-13, 2012. Berlin, Heidelberg: Springer, 2012: 746-760.
[71] XIAO J X, OWENS A, TORRALBA A. SUN3D: a database of big spaces reconstructed using SFM and object labels[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Dec 1-8, 2013. Washington: IEEE Computer Society, 2013: 1625-1632.
[72] SINGH A, SHA J, NARAYAN K S, et al. Bigbird: a large-scale 3D database of object instances[C]//Proceedings of the 2014 IEEE International Conference on Robotics and Automation, Hong Kong, China, May 31-Jun 7, 2014. Piscataway: IEEE, 2014: 509-516.
[73] MARTíNEZ-GóMEZ J, GARCíA-VAREA I, CAZORLA M, et al. ViDRILO: the visual and depth robot indoor localization with objects information dataset[J]. The International Journal of Robotics Research, 2015, 34(14): 1681-1687.
[74] SONG S R, LICHTENBERG S P, XIAO J X. SUN RGB-D: a RGB-D scene understanding benchmark suite[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 567-576.
[75] CHANG A, DAI A, FUNKHOUSER T, et al. Matterport3D: learning from RGB-D data in indoor environments[J]. arXiv: 1709.06158, 2017.
[76] CHEN X, GOLOVINSKIY A, FUNKHOUSER T. A benchmark for 3D mesh segmentation[J]. ACM Transactions on Graphics, 2009, 28(3): 1-12.
[77] YI L, KIM V G, CEYLAN D, et al. A scalable active framework for region annotation in 3D shape collections[J]. ACM Transactions on Graphics, 2016, 35(6): 1-12.
[78] ARMENI I, SENER O, ZAMIR A R, et al. 3D semantic parsing of large-scale indoor spaces[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 1534-1543.
[79] WANG C, HOU S, WEN C, et al. Semantic line framework-based indoor building modeling using backpacked laser scanning point cloud[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 143: 150-166.
[80] MUNOZ D, BAGNELL J A, VANDAPEL N, et al. Contextual classification with functional max-margin Markov networks[C]//Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Miami, Jun 20-25, 2009. Washington: IEEE Computer Society, 2009: 975-982.
[81] DE DEUGE M, QUADROS A, HUNG C, et al. Unsupervised feature learning for classification of outdoor 3D scans[C]//Proceedings of the 2013 Australasian Conference on Robitics and Automation, Sydney, Dec 2-4, 2013: 1-9.
[82] SERNA A, MARCOTEGUI B, GOULETTE F, et al. Paris-rue-Madame database: a 3D mobile laser scanner dataset for benchmarking urban detection, segmentation and classification methods[C]//Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods, Angers, Mar 6-8, 2014: 819-824.
[83] BRéDIF M, VALLET B, SERNA A, et al. TerraMobilita/IQmulus urban point cloud classification benchmark[C]//Workshop on Processing Large Geospatial Data, Cardiff, 2014.
[84] RIEMENSCHNEIDER H, BóDIS-SZOMORú A, WEISSENBERG J, et al. Learning where to classify in multi-view semantic segmentation[C]//LNCS 8693: Proceedings of the 13th European Conference on Computer Vision, Zurich, Sep 6-12, 2014. Berlin, Heidelberg: Springer, 2014: 516-532.
[85] GEHRUNG J, HEBEL M, ARENS M, et al. An approach to extract moving objects from MLS data using a volumetric background representation[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, 4: 107.
[86] GAIDON A, WANG Q, CABON Y, et al. Virtual worlds as proxy for multi-object tracking analysis[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Piscataway: IEEE, 2016: 4340-4349.
[87] HACKEL T, SAVINOV N, LADICKY L, et al. Semantic3D. net: a new large-scale point cloud classification benchmark[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017: 91-98.
[88] ROYNARD X, DESCHAUD J E, GOULETTE F. Paris-Lille-3D: a large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification[J]. The International Journal of Robotics Research, 2018, 37(6): 545-557.
[89] SONG X B, WANG P, ZHOU D F, et al. ApolloCar3D: a large 3D car instance understanding benchmark for autonomous driving[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 5452-5462.
[90] BEHLEY J, GARBADE M, MILIOTO A, et al. SemanticKITTI: a dataset for semantic scene understanding of LiDAR sequences[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 9296-9306.
[91] NIEMEYER J, ROTTENSTEINER F, SOERGEL U. Contextual classification of LiDAR data and building object detection in urban areas[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 87: 152-165.
[92] BOSCH M, FOSTER K, CHRISTIE G, et al. Semantic stereo for incidental satellite images[C]//Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision, Waikoloa Village, Jan 7-11, 2019. Piscataway: IEEE, 2019: 1524-1532.
[93] DONG Z, LIANG F, YANG B, et al. Registration of large-scale terrestrial laser scanner point clouds: a review and benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 163: 327-342.
[94] TATARCHENKO M, PARK J, KOLTUN V, et al. Tangent convolutions for dense prediction in 3D[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 3887-3896.
[95] KIRILLOV A, HE K, GIRSHICK R, et al. Panoptic segmentation[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 9404-9413.
[96] KIRILLOV A, GIRSHICK R, HE K, et al. Panoptic feature pyramid networks[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 6399-6408.
[97] LIU H Y, PENG C, YU C Q, et al. An end-to-end network for panoptic segmentation[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 6172-6181.
[98] YANG T, COLLINS M D, ZHU Y, et al. DeeperLab: single-shot image parser[J]. arXiv:1902.05093, 2019.
[99] CORDTS M, OMRAN M, RAMOS S, et al. The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 3213-3223.
[100] CAESAR H, UIJLINGS J, FERRARI V. COCO-Stuff: thing and stuff classes in context[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 1209-1218.
[101] DE GEUS D, MELETIS P, DUBBELMAN G. Fast panoptic segmentation network[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 1742-1749.
[102] ENGELMANN F, KONTOGIANNI T, LEIBE B. Dilated point convolutions: on the receptive field of point convo-lutions[J]. arXiv:1907.12046, 2019.
[103] SHEN Y R, FENG C, YANG Y Q, et al. Mining point cloud local structures by kernel correlation and graph pooling[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 4548-4557.
[104] JIANG L, ZHAO H S, LIU S, et al. Hierarchical point-edge interaction network for point cloud semantic segmentation[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 10433-10441.
[105] ZHAO Y, BIRDAL T, DENG H, et al. 3D point capsule networks[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1009-1018.
[106] CHERAGHIAN A, RAHMAN S, PETERSSON L. Zero-shot learning of 3D point cloud objects[C]//Proceedings of the 16th International Conference on Machine Vision Applications, Tokyo, May 27-31, 2019. Piscataway: IEEE, 2019: 1-6.
[107] LANDRIEU L, BOUSSAHA M. Point cloud oversegmentation with graph-structured deep metric learning[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Washington: IEEE Computer Society, 2019: 7440-7449.
[108] JARITZ M, GU J, SU H. Multi-view pointnet for 3D scene understanding[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Oct 27-28, 2019. Piscataway: IEEE, 2019: 3995-4003.
[109] LI Z, GAN Y K, LIANG X D, et al. LSTM-CF: unifying context modeling and fusion with LSTMs for RGB-D scene labeling[C]//LNCS 9906: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Berlin, Heidelberg: Springer, 2016: 541-557.
[110] LIU X Y, YAN M Y, BOHG J. MeteorNet: deep learning on dynamic 3D point cloud sequences[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 9246-9255.