Survey on 3D Reconstruction Methods Based on Visual Deep Learning

doi:10.3778/j.issn.1673-9418.2205054

Abstract

Abstract: In recent years, as one of the important tasks of computer vision, 3D reconstruction has received extensive attention. This paper focuses on the research progress of using deep learning to reconstruct the 3D shape of general objects in recent years. Taking the steps of 3D reconstruction by deep learning as the context, according to the data feature representation in the process of 3D reconstruction, it is divided into voxel, point cloud, surface mesh and implicit surface. Then, according to the number of inputting 2D images, it can be divided into single view 3D reconstruction and multi-view 3D reconstruction, which are subdivided according to the network architecture and the training mechanism they use. While the research progress of each category is discussed, the development prospects, advantages and disadvantages of each training method are analyzed. This paper studies the new hotspots in specific 3D reconstruction fields in recent years, such as 3D reconstruction of dynamic human bodies and 3D completion of incomplete geometric data, compares some key papers and summarizes the problems in these fields. Then this paper introduces the key application scenarios and parameters of 3D datasets at this stage. The development prospect of 3D reconstruction in specific application fields in the future is illustrated and analyzed, and the research direction of 3D reconstruction is prospected.

Key words: 3D reconstruction, visual deep learning, characterization reconstruction, geometric reconstruction, 3D completion, dynamic human reconstruction

摘要： 近年来，三维重建作为计算机视觉的重要任务之一，得到广泛的关注和深入的研究。重点分析近年来使用深度学习重建通用对象的三维形状的研究进展。以深度学习进行三维重建环节为脉络，根据三维重建过程中数据深度特征表示方法将三维重建研究分为体素、点云、曲面网格、隐式曲面四类。再根据输入二维图像的数目分为单视图三维重建和多视图三维重建两类，根据网络架构以及它们使用的训练机制进行具体细分，在讨论每个类别的研究进展的同时，分析每种训练方法的发展前景及优缺点。研究近年来在特定三维重建领域的新热点，例如动态人体三维重建和不完整几何数据的三维补全，对一些关键论文进行比较，总结了这些领域存在的问题。介绍现阶段的三维数据集的重点应用场景和参数。总结现阶段三维重建领域存在数据集缺失、模型训练复杂、缺少特定领域针对性识别等问题。对三维重建在未来的具体应用领域发展前景进行了例证分析，并对三维重建的研究方向进行了展望。

关键词: 三维重建, 视觉深度学习, 表征重建, 几何重建, 三维补全, 动态人体重建

LI Mingyang, CHEN Wei, WANG Shanshan, LI Jie, TIAN Zijian, ZHANG Fan. Survey on 3D Reconstruction Methods Based on Visual Deep Learning[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(2): 279-302.

李明阳, 陈伟, 王珊珊, 黎捷, 田子建, 张帆. 视觉深度学习的三维重建方法综述[J]. 计算机科学与探索, 2023, 17(2): 279-302.

References

[1] 龙霄潇, 程新景, 朱昊, 等. 三维视觉前沿进展[J]. 中国图象图形学报, 2021, 26(6): 1389-1428.
LONG X X, CHENG X J, ZHU H, et al. Recent progress in 3D vision[J]. Journal of Image and Graphics, 2021, 26(6): 1389-1428.
[2] 李永怀, 冯其波. 光学三维轮廓测量技术进展[J]. 激光与红外, 2005, 35(3): 143-147.
LI Y H, FENG Q B. Overview of optical 3D profile measurement[J]. Laser & Infrared, 2005, 35(3): 143-147.
[3] GARCIA E, DUGELAY J L, DELINGETTE H. Low cost 3D face acquisition and modeling[C]//Proceedings of the 2001 International Symposium on Information Technology, Las Vegas, Apr 2-4, 2001. Washington: IEEE Computer Society, 2001: 657-661.
[4] 丁少闻, 张小虎, 于起峰, 等. 非接触式三维重建测量方法综述[J]. 激光与光电子学进展, 2017, 54(7): 27-41.
DING S W, ZHANG X H, YU Q F, et al. Overview of non-contact 3D reconstruction measurement methods[J]. Laser & Optoelectronics Progress, 2017, 54(7): 27-41.
[5] 孙宇阳. 基于单幅图像的三维重建技术综述[J]. 北方工业大学学报, 2011, 23(1): 9-13.
SUN Y Y. A survey of 3D reconstruction based on single image[J]. Journal of North China University of Technology, 2011, 23(1): 9-13.
[6] 张文明, 刘彬, 李海滨. 基于双目视觉的三维重建中特征点提取及匹配算法的研究[J]. 光学技术, 2008, 34(2): 181-185.
ZHANG W M, LIU B, LI H B. Characteristic point extracts and the match algorithm based on the binocular vision in three dimensional reconstruction[J]. Optical Technique, 2008, 34(2): 181-185.
[7] EIGEN D, PUHRSCH C, FERGUS R. Depth map prediction from a single image using a multi-scale deep network[J]. arXiv:1406.2283, 2014.
[8] HAN X F, LAGA H, BENNAMOUN M. Image-based 3D object reconstruction: state-of-the-art and trends in the deep learning era[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(5): 1578-1604.
[9] SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 945-953.
[10] QI C R, SU H, NIE?NER M, et al. Volumetric and multi-view CNNs for object classification on 3D data[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 5648-5656.
[11] MATURANA D, SCHERER S. VoxNet: a 3D convolutional neural network for real-time object recognition[C]//Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems, Hamburg, Sep 28-Oct 2, 2015. Piscataway: IEEE, 2015: 922-928.
[12] WU Z R, SONG S R, KHOSLA A, et al. 3D ShapeNets: a deep representation for volumetric shapes[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 1912-1920.
[13] PESTOV V. On the geometry of similarity search: dimensionality curse and concentration of measure[J]. Information Processing Letters, 2000, 73(1/2): 47-51.
[14] LI Y Y, PIRK S, SU H, et al. FPNN: field probing neural networks for 3D data[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2016, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates, 2016: 307-315.
[15] WANG P S, LIU Y, GUO Y X, et al. O-CNN: octree-based convolutional neural networks for 3D shape analysis[J]. ACM Transactions on Graphics, 2017, 36(4): 72.
[16] QI C R, SU H, MO K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 77-85.
[17] SINHA A, BAI J, RAMANI K. Deep learning 3D shape surfaces using geometry images[C]//LNCS 9910: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 223-240.
[18] RAKOTOSAONA M J, GUERRERO P, AIGERMAN N, et al. Learning Delaunay surface elements for mesh reconstruction[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 22-31.
[19] MESCHEDER L M, OECHSLE M, NIEMEYER M, et al. Occupancy networks: learning 3D reconstruction in function space[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 4460-4470.
[20] ZHENG Z R, YU T, DAI Q H, et al. Deep implicit templates for 3D shape representation[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 1429-1439.
[21] ERLER P, GUERRERO P, OHRHALLINGER S, et al. Points2Surf: learning implicit surfaces from point cloud patches[J]. arXiv:2007.10453, 2020.
[22] GIRDHAR R, FOUHEY D F, RODRIGUEZ M, et al. Learning a predictable and generative vector representation for objects[C]//LNCS 9910: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 484-499.
[23] CHOY C B, XU D F, GWAK J Y, et al. 3D-R2N2: a unified approach for single and multi-view 3D object reconstruction[C]//LNCS 9912: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 628-644.
[24] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[25] TATARCHENKO M, DOSOVITSKIY A, BROX T. Octree generating networks: efficient convolutional architectures for high-resolution 3D outputs[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2107-2115.
[26] REZENDE D J, ESLAMI S M A, MOHAMED S, et al. Unsupervised learning of 3D structure from images[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2016, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates, 2016: 4997-5005
[27] STUTZ D, GEIGER A. Learning 3D shape completion from laser scan data with weak supervision[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 1955-1964.
[28] YAN X C, YANG J M, YUMER E, et al. Perspective transformer nets: learning single-view 3D object reconstruction without 3D supervision[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2016, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates, 2016: 1696-1704.
[29] WU J J, ZHANG C K, XUE T F, et al. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2016, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates 2016: 82-90.
[30] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2014, Montreal Dec 8-13, 2014. Red Hook: Curran Associates, 2014: 2672-2680.
[31] WU J J, WANG Y F, XUE T F, et al. MarrNet: 3D shape reconstruction via 2.5 D sketches[J]. arXiv:1711.03129, 2017.
[32] JI M Q, GALL J, ZHENG H T, et al. SurfaceNet: an end-to-end 3D neural network for multiview stereopsis[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2326-2334.
[33] KAR A, H?NE C, MALIK J. Learning a multi-view stereo machine[J]. arXiv:1708.05375, 2017.
[34] 赵晨园, 李文新, 张庆熙. 双目视觉的立体匹配算法研究进展[J]. 计算机科学与探索, 2020, 14(7): 1104-1113.
ZHAO C Y, LI W X, ZHANG Q X. Research and development of binocular stereo matching algorithm[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(7): 1104-1113.
[35] HUANG P H, MATZEN K, KOPF J, et al. DeepMVS: learning multi-view stereopsis[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 2821-2830.
[36] YAO Y, LUO Z X, S LI W, et al. MVSNet: depth inference for unstructured multi-view stereo[C]//LNSC 11212: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 785-801.
[37] KNAPITSCH A, PARK J, ZHOU Q Y, et al. Tanks and temples: benchmarking large-scale scene reconstruction[J]. ACM Transactions on Graphics, 2017, 36(4): 78.
[38] 隋婧, 金伟其. 双目立体视觉技术的实现及其进展[J]. 电子技术应用, 2004, 30(10): 4-6.
SUI J, JIN W Q. Realization and progress of binocular stereo vision technology[J]. Application of Electronic Technique, 2004, 30(10): 4-6.
[39] YAO Y, LUO Z X, LI S W, et al. Recurrent MVSNet for high-resolution multi-view stereo depth inference[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 5525-5534.
[40] CHO K, VAN MERRI?NBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv:1406.1078, 2014.
[41] FAN H Q, SU H, GUIBAS L J. A point set generation network for 3D object reconstruction from a single image[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 2463-2471.
[42] ACHLIOPTAS P, DIAMANTI O, MITLIAGKAS I, et al. Learning representations and generative models for 3D point clouds[C]//Proceedings of the 35th International Conference on Machine Learning, Stockholmsm?ssan, Jul 10-15, 2018: 40-49.
[43] KINGMA D P, WELLING M. Auto-encoding variational Bayes[J]. arXiv:1312.6114, 2013.
[44] YANG Y Q, FENG C, SHEN Y R, et al. FoldingNet: point cloud auto-encoder via deep grid deformation[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 206-215.
[45] CHANG C C, LIN C J. LIBSVM: a library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 27.
[46] YANG G D, HUANG X, HAO Z K, et al. PointFlow: 3D point cloud generation with continuous normalizing flows[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 4540-4549.
[47] KIMURA T, MATSUBARA T, UEHARA K. ChartPointFlow for topology-aware 3D point cloud generation[C]//Proceedings of the 29th ACM International Conference on Multimedia, Oct 20-24, 2021. New York: ACM, 2021: 1396-1404.
[48] KLOKOV R, BOYER E, VERBEEK J. Discrete point flow networks for efficient point cloud generation[C]//LNCS 12368: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 694-710.
[49] CHEN R, HAN S F, XU J, et al. Point-based multi-view stereo network[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1538-1547.
[50] LUO K, GUAN T, JU L, et al. P-MVSNet: learning patch-wise matching confidence aggregation for multi-view stereo[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 10452-10461.
[51] XIE J W, XU Y F, ZHENG Z L, et al. Generative PointNet: deep energy-based learning on unordered point sets for 3D generation, reconstruction and classification[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 14976-14985.
[52] SPUREK P, WINCZOWSKI S, TABOR J, et al. Hypernetwork approach to generating point clouds[J]. arXiv:2003. 00802, 2020.
[53] CAI R J, YANG G D, AVERBUCH-ELOR H, et al. Learning gradient fields for shape generation[C]//LNCS 12348: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 364-381.
[54] GRATHWOHL W, CHEN R T, BETTENCOURT J, et al. FFJORD: free-form continuous dynamics for scalable reversible generative models[J]. arXiv:1810.01367, 2018.
[55] LUO S T, HU W. Diffusion probabilistic models for 3D point cloud generation[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 2837-2845.
[56] GEYER C J. Practical Markov chain Monte Carlo[J]. Statistical Science, 1992, 7(4): 473-483.
[57] GROUEIX T, FISHER M, KIM V G, et al. A Papier-Maché approach to learning 3D surface generation[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 216-224.
[58] WANG N Y, ZHANG Y D, LI Z W, et al. Pixel2Mesh: generating 3D mesh models from single RGB images[C]//LNCS 11215: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 52-67.
[59] KONG C, LIN C, LUCEY S. Using locally corresponding CAD models for dense 3D reconstructions from a single image[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 5603-5611.
[60] PATI Y C, REZAIIFAR R, KRISHNAPRASAD P S. Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition[C]//Proceedings of the 27th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, Nov 1-3, 1993. Piscataway: IEEE, 1993: 40-44.
[61] CHEN Z Q, ZHANG H. Learning implicit fields for generative shape modeling[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 5939-5948.
[62] NIEMEYER M, MESCHEDER L, OECHSLE M, et al. Differentiable volumetric rendering: learning implicit 3D representations without 3D supervision[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 3501-3512.
[63] LI T M, AITTALA M, DURAND F, et al. Differentiable Monte Carlo ray tracing through edge sampling[J]. ACM Transactions on Graphics, 2018, 37(6): 222.
[64] JIANG C, SUD A, MAKADIA A, et al. Local implicit grid representations for 3D scenes[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 6000-6009.
[65] PROSZEWSKA M, MAZUR M, TRZCI?SKI T, et al. HyperCube: implicit field representations of voxelized 3D models[J]. arXiv:2110.05770, 2021.
[66] HUANG Z T, YU Y K, XU J W, et al. PF-Net: point fractal network for 3D point cloud completion[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 7659-7667.
[67] XIE H Z, YAO H X, SUN X S, et al. Pix2Vox: context-aware 3D reconstruction from single and multi-view images[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 2690-2698.
[68] CHIBANE J, ALLDIECK T, PONS-MOLL G. Implicit functions in feature space for 3D shape reconstruction and completion[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 6968-6979.
[69] MITRA N J, WAND M, ZHANG H, et al. Structure-aware shape processing[C]//Proceedings of the SIGGRAPH Asia 2013, Hong Kong, China, Nov 19-22, 2013. New York: ACM, 2013: 1-20.
[70] LI J, XU K, CHAUDHURI S, et al. GRASS: generative recursive autoencoders for shape structures[J]. ACM Transactions on Graphics, 2017, 36(4): 52.
[71] HAN X G, GAO C, YU Y Z. DeepSketch2Face: a deep learning based sketching system for 3D face and caricature modeling[J]. ACM Transactions on Graphics, 2017, 36(4): 126.
[72] GAO L, YANG J, WU T, et al. SDM-NET: deep generative network for structured deformable mesh[J]. ACM Transactions on Graphics, 2019, 38(6): 243.
[73] WU Z, WANG X, LIN D, et al. Structure-aware generative network for 3D-shape modeling[J]. arXiv:1808.03981, 2018.
[74] SITZMANN V, ZOLLH?FER M, WETZSTEIN G. Scene representation networks: continuous 3D-structure-aware neural scene representations[J]. arXiv:1906.01618, 2019.
[75] MO K, GUERRERO P, YI L, et al. StructureNet: hierarchical graph networks for 3D shape generation[J]. arXiv:1908. 00575, 2019.
[76] MO K, GUERRERO P, YI L, et al. StructEdit: learning structural shape variations[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 8856-8865.
[77] CHEN Z, TAGLIASACCHI A, ZHANG H. BSP-Net: generating compact meshes via binary space partitioning[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 42-51.
[78] CHEN Z, ZHANG Y D, GENOVA K, et al. Multiresolution deep implicit functions for 3D shape representation[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Oct 10-17, 2021. Piscataway: IEEE, 2021: 13067-13076.
[79] WU R D, ZHUANG Y X, XU K, et al. PQ-NET: a generative part Seq2Seq network for 3D shapes[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 826-835.
[80] LI J, NIU C J, XU K. Learning part generation and assembly for structure-aware shape synthesis[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, the 32nd Innovative Applications of Artificial Intelligence Conference, the 10th AAAI Symposium on Educational Advances in Artificial Intelligence, New York, Feb 7-12, 2020. Menlo Park: AAAI, 2020: 11362-11369.
[81] ECKART B, YUAN W T. LIU C, et al. Self-supervised learning on 3D point clouds by learning discrete generative models[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 8248-8257.
[82] ZIVKOVIC Z. Improved adaptive Gaussian mixture model for background subtraction[C]//Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, Aug 23-26, 2004. Washington: IEEE Computer Society, 2004: 28-31.
[83] DAI A, QI C R, NIE?NER M. Shape completion using 3D-Encoder-Predictor CNNs and shape synthesis[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 5868-5877.
[84] DARABI S, SHECHTMAN E, BARNES C, et al. Image melding: combining inconsistent images using patch-based synthesis[J]. ACM Transactions on Graphics, 2012, 31(4): 82.
[85] HAN X G, LI Z, HUANG H B, et al. High-resolution shape completion using deep neural networks for global structure and local geometry inference[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 85-93.
[86] CHENG J, DONG L, LAPATA M. Long short-term memory-networks for machine reading[J]. arXiv:1601.06733, 2016.
[87] YUAN W T, KHOT T, HELD D, et al. PCN: point completion network[C]//Proceedings of the 2018 International Conference on 3D Vision, Verona, Sep 5-8, 2018. Washington: IEEE Computer Society, 2018: 728-737.
[88] WEN X, LI T, HAN Z, et al. Point cloud completion by skip-attention network with hierarchical folding[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 1939-1948.
[89] WEN X, XIANG P, HAN Z, et al. PMP-Net: point cloud completion by learning multi-step point moving paths[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 7443-7452.
[90] MITCHELL J S. Geometric shortest paths and network optimization[M]//SACK J R, URRUTIA J. Handbook of Computational Geometry. New York: Elsevier Science Inc., 2000.
[91] WEN X, HAN Z, CAO Y P, et al. Cycle4Completion: unpaired point cloud completion using cycle transformation with missing region coding[C]//Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 13080-13089.
[92] PARK J J, FLORENCE P, STRAUB J, et al. DeepSDF: learning continuous signed distance functions for shape representation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 165-174.
[93] LIU S L, GUO H X, PAN H, et al. Deep implicit moving least-squares functions for 3D reconstruction[C]//Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 1788-1797.
[94] LANCASTER P, SALKAUSKAS K. Surfaces generated by moving least squares methods[J]. Mathematics of Computation, 1981, 37(155): 141-158.
[95] GENOVA K, COLE F, SUD A, et al. Deep structured implicit functions[J]. arXiv:1912.06126, 2019.
[96] MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[C]//LNCS 12346: Proceedings of the 16th European Conference on Computer Vision, Amsterdam, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 405-421.
[97] INNMANN M, ZOLLH?FER M, NIE?NER M, et al. Volume-Deform: real-time volumetric non-rigid reconstruction[C]//LNCS 9912: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 362-379.
[98] CORONA E, PUMAROLA A, ALENYA G, et al. SMPLicit: topology-aware generative model for clothed people[C]//Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 11875-11885.
[99] SAITO S, HUANG Z, NATSUME R, et al. PIFu: pixel-aligned implicit function for high-resolution clothed human digitization[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 2304-2314.
[100] BOGO F, KANAZAWA A, LASSNER C, et al. Keep it SMPL: automatic estimation of 3D human pose and shape from a single image[C]//LNCS 9909: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 561-578.
[101] LOPER M, MAHMOOD N, ROMERO J, et al. SMPL: a skinned multi-person linear model[J]. ACM Transactions on Graphics, 2015, 34(6): 248.
[102] SAITO S, YANG J L, MA Q L, et al. SCANimate: weakly supervised learning of skinned clothed Avatar networks[C]//Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 2886-2897.
[103] BULDYREV S, GOLDBERGER A, HAVLIN S, et al. Long-range correlation properties of coding and noncoding DNA sequences: GenBank analysis[J]. Physical Review E, 1995, 51(5): 5084 -5091.
[104] MIHAJLOVIC M, ZHANG Y, BLACK M J, et al. LEAP: learning articulated occupancy of people[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 10461-10471.
[105] ZHU X, LIU X, LEI Z, et al. Face alignment in full pose range: a 3D total solution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 41(1): 78-92.
[106] BLANZ V, VETTER T. Face recognition based on fitting a 3D morphable model[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(9): 1063-1074.
[107] FENG Y, WU F, SHAO X H, et al. Joint 3D face reconstruction and dense alignment with position map regression network[C]//LNCS 11218: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 557-574.
[108] DENG Y, YANG J L, XU S C, et al. Accurate 3D face reconstruction with weakly-supervised learning: from single image to image set[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 285-295.
[109] GENOVA K, COLE F, MASCHINOT A, et al. Unsupervised training for 3D morphable model regression[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 8377-8386.
[110] TEWARI A, ZOLLHOFER M, KIM H, et al. MoFA: model-based deep convolutional face autoencoder for unsupervised monocular reconstruction[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 1274-1283.
[111] ZHU W B, WU H T, CHEN Z Y, et al. ReDA: reinforced differentiable attribute for 3D face reconstruction[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 4957-4966.
[112] DAI A, NIE?NER M. Scan2Mesh: from unstructured range scans to 3D meshes[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 5574-5583.
[113] GECER B, S. PLOUMPIS, KOTSIA I, et al. GANFIT: generative adversarial network fitting for high fidelity 3D face reconstruction[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1155-1164.
[114] KARRAS T, LAINE S, AILA T. A style-based generator architecture for generative adversarial networks[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 4401-4410.
[115] PIAO J T, SUN K Q, WANG Q, et al. Inverting generative adversarial renderer for face reconstruction[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 15619-15628.
[116] LUO H W, NAGANO K, KUNG H W, et al. Normalized Avatar synthesis using StyleGAN and perceptual refinement[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, Jun 19-25, 2021. Piscataway: IEEE, 2021: 11662-11672.
[117] SILBERMAN N, HOIEM D, KOHLI P, et al. Indoor segmentation and support inference from RGBD images[C]//LNCS 7576: Proceedings of the 12th European Conference on Computer Vision, Florence, Oct 7-13, 2012. Cham: Springer, 2012: 746-760.
[118] LIM J J, PIRSIAVASH H, TORRALBA A. Parsing IKEA objects: fine pose estimation[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Dec 1-8, 2013. Washington: IEEE Computer Society, 2013: 2992-2999.
[119] XIANG Y, MOTTAGHI R, SAVARESE S. Beyond PASCAL: a benchmark for 3D object detection in the wild[C]//Proceedings of the 2014 IEEE Winter Conference on Applications of Computer Vision, Steamboat Springs, Mar 24-26, 2014. Washington: IEEE Computer Society, 2014: 75-82.
[120] CHANG A X, FUNKHOUSER T, GUIBAS L, et al. Shape-Net: an information-rich 3D model repository[J]. arXiv: 1512.03012, 2015.
[121] DAI A, CHANG A X, SAVVA M, et al. ScanNet: richly-annotated 3D reconstructions of indoor scenes[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 5828-5839.
[122] SONG S R, YU F, ZENG A, et al. Semantic scene completion from a single depth image[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 190-198.
[123] ARMENI I, SAX S, ZAMIR A R, et al. Joint 2D-3D-semantic data for indoor scene understanding[J]. arXiv:1702. 01105, 2017.
[124] SAINT A, AHMED E, SHABAYEK A E R, et al. 3DBodyTex: textured 3D body dataset[C]//Proceedings of the 2018 International Conference on 3D Vision, Verona, Sep 5-8, 2018. Washington: IEEE Computer Society, 2018: 495-504.
[125] GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: the KITTI dataset[J]. International Journal of Robotics Research, 2013, 32(11): 1231-1237.
[126] SCH?PS T, SCH?NBERGER J L, GALLIANI S, et al. A multi-view stereo benchmark with high-resolution images and multi-camera videos[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 2538-2547.
[127] JENSEN R R, DAHL A L, VOGIATZIS G, et al. Large scale multi-view stereopsis evaluation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Jun 23-28, 2014. Washington: IEEE Computer Society, 2014: 406-413.
[128] PAYSAN P, KNOTHE R, B. AMBERG, et al. A 3D face model for pose and illumination invariant face recognition[C]//Proceedings of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, Genova, Sep 2-4, 2009. Washington: IEEE Computer Society, 2009: 296-301.
[129] BOOTH J, ROUSSOS A, PONNIAH A, et al. Large scale 3D morphable models[J]. International Journal of Computer Vision, 2018, 126(2): 233-254.
[130] SCHMIDT R M, SINGH K. MeshMixer: an interface for rapid mesh composition[C]//Proceedings of the 2010 International Conference on Computer Graphics and Interactive Techniques, Los Angeles, Jul 26-30, 2010. New York: ACM, 2010.
[131] LU Y, LIU C, KEVIN I, et al. Digital twin-driven smart manufacturing: connotation, reference model, applications and research issues[J]. Robotics and Computer-Integrated Manufacturing, 2020, 61: 101837.
[132] 唐维军, 徐琨, 柳有权, 等. 自动驾驶汽车虚拟测试中的树木点云生成方法[J]. 计算机工程与应用, 2021, 57(11): 185-192.
TANG W J, XU K, LIU Y Q, et al. Method for generating three-dimensional point cloud of trees in autonomous vehicle virtual testing[J]. Computer Engineering and Applications, 2021, 57(11): 185-192.
[133] DURRANT-WHYTE H, BAILEY T. Simultaneous localization and mapping: part I[J]. IEEE Robotics & Automation Magazine, 2006, 13(2): 99-110.
[134] SCHÖNBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 4104-4113.