Journal of Frontiers of Computer Science and Technology ›› 2024, Vol. 18 ›› Issue (4): 831-860.DOI: 10.3778/j.issn.1673-9418.2305016

• Frontiers·Surveys • Previous Articles     Next Articles

Review of Research on 3D Reconstruction of Dynamic Scenes

SUN Shuifa, TANG Yongheng, WANG Ben, DONG Fangmin, LI Xiaolong, CAI Jiacheng, WU Yirong   

  1. 1. Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering, Three Gorges University, Yichang, Hubei 443002, China
    2. College of Computer and Information Technology, Three Gorges University, Yichang, Hubei 443002, China
    3. College of Information Science and Technology, Hangzhou Normal University, Hangzhou 310000, China
    4. College of Economics and Management, Three Gorges University, Yichang, Hubei 443002, China
    5. Institute of Advanced Studies in Humanities and Social Sciences, Beijing Normal University, Zhuhai, Guangdong 519087, China
  • Online:2024-04-01 Published:2024-04-01

动态场景的三维重建研究综述

孙水发,汤永恒,王奔,董方敏,李小龙,蔡嘉诚,吴义熔   

  1. 1. 三峡大学 水电工程智能视觉监测湖北省重点实验室,湖北 宜昌 443002
    2. 三峡大学 计算机与信息学院,湖北 宜昌 443002
    3. 杭州师范大学 信息科学与技术学院,杭州 310000
    4. 三峡大学 经济与管理学院,湖北 宜昌 443002
    5. 北京师范大学 人文和社会科学高等研究院,广东 珠海 519087

Abstract: As static scene 3D reconstruction algorithms become more mature, dynamic scene 3D reconstruction has become a hot and challenging research topic in recent years. Existing static scene 3D reconstruction algorithms have good reconstruction results for stationary objects. However, when objects in the scene undergo deformation or relative motion, their reconstruction results are not ideal. Therefore, developing research on 3D reconstruction of dynamic scenes is essential. This paper first introduces the related concepts and basic knowledge of 3D reconstruction, as well as the research classification and current status of static and dynamic scene 3D reconstruction. Then, the latest research progress on dynamic scene 3D reconstruction is comprehensively summarized, and the reconstruction algorithms are classified into dynamic 3D reconstruction based on RGB data sources and dynamic 3D reconstruction based on RGB-D data sources. RGB data sources can be further divided into template based dynamic 3D reconstruction, non rigid motion recovery structure based dynamic 3D reconstruction, and learning based dynamic 3D reconstruction under RGB data sources. The RGB-D data source mainly summarizes dynamic 3D reconstruction based on learning, with typical examples introduced and compared. The applications of dynamic scene 3D reconstruction in medical, intelligent manufacturing, virtual reality and augmented reality, and transportation fields are also discussed. Finally, future research directions for dynamic scene 3D reconstruction are proposed, and an outlook on the research progress in this rapidly developing field is presented.

Key words: dynamic scene 3D reconstruction, template prior, motion recovery structure, deep learning

摘要: 随着静态场景三维重建算法的不断成熟,动态场景三维重建算法成为近年来的研究热点和研究难点。现有的静态场景三维重建算法对静止的对象有较好的重建效果,一旦场景中对象出现变形或者是相对运动,其重建效果不太理想,因此发展对动态场景的三维重建研究工作是相当重要的。简要介绍三维重建的相关概念及基本知识、静态场景三维重建和动态场景三维重建的研究分类及研究现状;全面总结了动态场景三维重建研究最新进展,将动态场景三维重建按照基于RGB数据源的动态三维重建和基于RGB-D数据源的动态三维重建进行分类,其中RGB数据源下又可划分为基于模板的动态三维重建、基于非刚性运动恢复结构的动态三维重建和RGB数据源下基于学习的动态三维重建,RGB-D数据源下主要总结归纳基于学习的动态三维重建,对各类典型重建算法进行了介绍和对比分析;介绍了动态场景三维重建在医学、智能制造、虚拟现实与增强现实、交通等领域的应用;提出了动态场景三维重建的未来研究方向,并对这个快速发展领域中的各个方向研究进行了展望。

关键词: 动态场景三维重建, 模板先验, 运动恢复结构, 深度学习