Journal of Frontiers of Computer Science and Technology ›› 2023, Vol. 17 ›› Issue (2): 279-302.DOI: 10.3778/j.issn.1673-9418.2205054

• Frontiers·Surveys • Previous Articles     Next Articles

Survey on 3D Reconstruction Methods Based on Visual Deep Learning

LI Mingyang, CHEN Wei, WANG Shanshan, LI Jie, TIAN Zijian, ZHANG Fan   

  1. 1. School of Computer Science & Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116, China
    2. Engineering Research Center of Mine Digitalization of Ministry of Education, China University of Mining and Technology, Xuzhou, Jiangsu 221116, China
    3. School of Mechanical Electronic and Information Engineering, China University of Mining and Technology (Beijing), Beijing 100083, China
  • Online:2023-02-01 Published:2023-02-01

视觉深度学习的三维重建方法综述

李明阳,陈伟,王珊珊,黎捷,田子建,张帆   

  1. 1. 中国矿业大学 计算机科学与技术学院,江苏 徐州 221116
    2. 中国矿业大学 矿山数字化教育部工程研究中心,江苏 徐州 221116
    3. 中国矿业大学(北京) 机电与信息工程学院,北京 100083

Abstract: In recent years, as one of the important tasks of computer vision, 3D reconstruction has received extensive attention. This paper focuses on the research progress of using deep learning to reconstruct the 3D shape of general objects in recent years. Taking the steps of 3D reconstruction by deep learning as the context, according to the data feature representation in the process of 3D reconstruction, it is divided into voxel, point cloud, surface mesh and implicit surface. Then, according to the number of inputting 2D images, it can be divided into single view 3D reconstruction and multi-view 3D reconstruction, which are subdivided according to the network architecture and the training mechanism they use. While the research progress of each category is discussed, the development prospects, advantages and disadvantages of each training method are analyzed. This paper studies the new hotspots in specific 3D reconstruction fields in recent years, such as 3D reconstruction of dynamic human bodies and 3D completion of incomplete geometric data, compares some key papers and summarizes the problems in these fields. Then this paper introduces the key application scenarios and parameters of 3D datasets at this stage. The development prospect of 3D reconstruction in specific application fields in the future is illustrated and analyzed, and the research direction of 3D reconstruction is prospected.

Key words: 3D reconstruction, visual deep learning, characterization reconstruction, geometric reconstruction, 3D completion, dynamic human reconstruction

摘要: 近年来,三维重建作为计算机视觉的重要任务之一,得到广泛的关注和深入的研究。重点分析近年来使用深度学习重建通用对象的三维形状的研究进展。以深度学习进行三维重建环节为脉络,根据三维重建过程中数据深度特征表示方法将三维重建研究分为体素、点云、曲面网格、隐式曲面四类。再根据输入二维图像的数目分为单视图三维重建和多视图三维重建两类,根据网络架构以及它们使用的训练机制进行具体细分,在讨论每个类别的研究进展的同时,分析每种训练方法的发展前景及优缺点。研究近年来在特定三维重建领域的新热点,例如动态人体三维重建和不完整几何数据的三维补全,对一些关键论文进行比较,总结了这些领域存在的问题。介绍现阶段的三维数据集的重点应用场景和参数。总结现阶段三维重建领域存在数据集缺失、模型训练复杂、缺少特定领域针对性识别等问题。对三维重建在未来的具体应用领域发展前景进行了例证分析,并对三维重建的研究方向进行了展望。

关键词: 三维重建, 视觉深度学习, 表征重建, 几何重建, 三维补全, 动态人体重建