计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (5): 1117-1127.DOI: 10.3778/j.issn.1673-9418.2010008

• 人工智能 • 上一篇    下一篇

由形状结构和位姿特征学习的稠密点云重建

杨永兆, 张玉金(), 张立军   

  1. 上海工程技术大学 电子电气工程学院,上海 201620
  • 收稿日期:2020-10-09 修回日期:2021-03-12 出版日期:2022-05-01 发布日期:2022-05-19
  • 通讯作者: + E-mail: yjzhang@sues.edu.cn
  • 作者简介:杨永兆(1993—),男,江苏南京人,硕士研究生,主要研究方向为计算机视觉。
    张玉金(1982—),男,安徽滁州人,博士,副教授,硕士生导师,主要研究方向为计算机视觉、多媒体内容安全、图像处理、模式识别等。
    张立军(1974—),男,山东东营人,博士,讲师,主要研究方向为机器学习、人工智能等。
  • 基金资助:
    上海市自然科学基金(17ZR1411900);上海市科委重点项目(18511101600)

Dense Point Cloud Reconstruction by Shape and Pose Features Learning

YANG Yongzhao, ZHANG Yujin(), ZHANG Lijun   

  1. School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China
  • Received:2020-10-09 Revised:2021-03-12 Online:2022-05-01 Published:2022-05-19
  • About author:YANG Yongzhao, born in 1993, M.S. candidate. His research interest is computer vision.
    ZHANG Yujin, born in 1982, Ph.D., associate professor, M.S. supervisor. His research interests include computer vision, multimedia content security, image processing, pattern recognition, etc.
    ZHANG Lijun, born in 1974, Ph.D., lecturer. His research interests include machine learning, artificial intelligence, etc.
  • Supported by:
    Natural Science Foundation of Shanghai(17ZR1411900);Key Projects of Science and Technology Commission of Shanghai(18511101600)

摘要:

作为高分辨率三维重建的方法之一,从单张图像生成稠密三维点云在计算机视觉领域中一直有着较高的关注度。针对以往这个方法中大多关注目标单一特征信息和使用样本数据量大的问题,提出一个基于特征多样性的多阶段重建稠密点云网络。该网络模型是由第一阶段的3D重建网络和第二阶段的点云处理网络两部分两阶段组成。第一阶段的3D重建网络在融合2D图像目标形状特征与3D点云位姿特征基础上,实现从单张图像重建稀疏点云操作。第二阶段的点云处理网络在稀疏点云基础上提取全局特征和局部特征,通过融合全局和局部点特征增加点的稠密度,得到高分辨率稠密点云。运用深度学习微调技术组合两阶段网络形成一个能从单张图像生成稠密点云的端到端网络。该方法在合成和真实世界数据集上通过大量实验定量和定性分析,结果表明,该方法平均CD值为0.006 98,EMD值为2 823.53,结果优于一些现有方法,且点云重建效果较好。

关键词: 三维重建, 稠密点云, 特征多样性, 多阶段重建, 微调技术

Abstract:

As one of the methods of high-resolution 3D reconstruction, generating dense 3D point clouds from a single image has always been of high interest in the field of computer vision. In view of most methods focusing on the single feature information of the target and the large amount of sample data used, the method of a multi-stage reconstruction of dense point cloud network based on feature diversity is proposed, which is composed of the first stage of the 3D reconstruction network and the second stage of the point cloud processing network. The 3D reconstruction network can reconstruct sparse point cloud from a single image based on the fusion of 2D image target shape features and 3D point cloud pose features. The second-stage point cloud processing network extracts global and local features based on sparse point clouds, and increases the density of points by fusing global and local point features to obtain high-resolution dense point clouds. Deep learning fine-tuning technology is used to combine two networks to form an end-to-end network that can generate dense point clouds from a single image. The method in this paper is quantitatively and qualitatively analyzed through a large number of experiments on synthetic and real-world datasets. The results show that the average CD value of this method is 0.00698, and the EMD value is 2823.53. The result is better than some existing methods, and the point cloud reconstruction effect is better.

Key words: 3D reconstruction, dense point cloud, feature diversity, multi-stage reconstruction, fine-tuning

中图分类号: