Abstract:

As one of the methods of high-resolution 3D reconstruction, generating dense 3D point clouds from a single image has always been of high interest in the field of computer vision. In view of most methods focusing on the single feature information of the target and the large amount of sample data used, the method of a multi-stage reconstruction of dense point cloud network based on feature diversity is proposed, which is composed of the first stage of the 3D reconstruction network and the second stage of the point cloud processing network. The 3D reconstruction network can reconstruct sparse point cloud from a single image based on the fusion of 2D image target shape features and 3D point cloud pose features. The second-stage point cloud processing network extracts global and local features based on sparse point clouds, and increases the density of points by fusing global and local point features to obtain high-resolution dense point clouds. Deep learning fine-tuning technology is used to combine two networks to form an end-to-end network that can generate dense point clouds from a single image. The method in this paper is quantitatively and qualitatively analyzed through a large number of experiments on synthetic and real-world datasets. The results show that the average CD value of this method is 0.00698, and the EMD value is 2823.53. The result is better than some existing methods, and the point cloud reconstruction effect is better.

Key words: 3D reconstruction, dense point cloud, feature diversity, multi-stage reconstruction, fine-tuning

摘要：

作为高分辨率三维重建的方法之一,从单张图像生成稠密三维点云在计算机视觉领域中一直有着较高的关注度。针对以往这个方法中大多关注目标单一特征信息和使用样本数据量大的问题,提出一个基于特征多样性的多阶段重建稠密点云网络。该网络模型是由第一阶段的3D重建网络和第二阶段的点云处理网络两部分两阶段组成。第一阶段的3D重建网络在融合2D图像目标形状特征与3D点云位姿特征基础上,实现从单张图像重建稀疏点云操作。第二阶段的点云处理网络在稀疏点云基础上提取全局特征和局部特征,通过融合全局和局部点特征增加点的稠密度,得到高分辨率稠密点云。运用深度学习微调技术组合两阶段网络形成一个能从单张图像生成稠密点云的端到端网络。该方法在合成和真实世界数据集上通过大量实验定量和定性分析,结果表明,该方法平均CD值为0.006 98,EMD值为2 823.53,结果优于一些现有方法,且点云重建效果较好。

关键词: 三维重建, 稠密点云, 特征多样性, 多阶段重建, 微调技术