Co-segmentation of 3D Point Cloud Shape Clusters Based on Weakly Supervised Learning

doi:10.3778/j.issn.1673-9418.2012036

Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (9): 2121-2131.DOI: 10.3778/j.issn.1673-9418.2012036

• Graphics and Image • Previous Articles Next Articles

Co-segmentation of 3D Point Cloud Shape Clusters Based on Weakly Supervised Learning

YANG Jun¹^,², LEI Xiwen¹^,⁺()

1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
2. Faculty of Geomatics, Lanzhou Jiaotong University, Lanzhou 730070, China

Received:2020-12-10 Revised:2021-04-06 Online:2022-09-01 Published:2021-04-25
About author:YANG Jun, born in 1973, Ph.D., professor. His research interests include deep learning, computer graphics, remote sensing image interpretation, etc.
LEI Xiwen, born in 1990, M.S. candidate. His research interests include computer graphics and pattern recognition.
Supported by:
National Natural Science Foundation of China(61862039);Science and Technology Program of Gansu Province(20JR5RA429);Talent Innovation and Entrepreneurship Project of Lanzhou(2020-RC-22);Project of Tianyou Innovation Team of Lanzhou Jiaotong University(TY202002)

弱监督学习下的三维点云模型簇协同分割

杨军¹^,², 雷喜文¹^,⁺()

1.兰州交通大学电子与信息工程学院,兰州 730070
2.兰州交通大学测绘与地理信息学院,兰州 730070

通讯作者: + E-mail: 314702467@qq.com
作者简介:杨军（1973—）,男,宁夏吴忠人,博士,教授,主要研究方向为深度学习、计算机图形学、遥感影像解译等。
雷喜文（1990—）,男,甘肃庆阳人,硕士研究生,主要研究方向为计算机图形学、模式识别。
基金资助:
国家自然科学基金(61862039);甘肃省科技计划资助(20JR5RA429);兰州市人才创新创业项目(2020-RC-22);兰州交通大学天佑创新团队项目(TY202002)

Abstract

Abstract:

With the rapid development of 3D acquisition technology, point cloud data have gradually become one of the basic data formats to represent 3D shapes, which can retain more raw geometric information of the shape in 3D space. However, in the field of 3D point cloud shape segmentation, most deep learning network architectures rely on high-quality labeled data, which leads to high training cost. Therefore, in order to solve the problem of co-segmen-tation of 3D shape clusters by using training samples with a small number of labeled points, a consistent segmenta-tion of 3D point cloud shape clusters based on weakly supervised learning method is proposed. Firstly, the local neighborhood graph between points is established by K-nearest neighbor algorithm. Then, the feature of the point cloud model is extracted by local convolution method, and similar component matrices are constructed by using the extracted component features. Finally, an energy function reverse iteration is used to optimize the network weights to obtain the consistent segmentation results of the shape clusters. Experimental results show that the segmentation accuracy of this algorithm is 85.0% on ShapeNet Parts. Compared with the existing supervised learning algorithms, when the number of labeled points in the training samples is reduced to 10%, the proposed algorithm can still achieve similar or even better results, and compared with the current mainstream weakly monitoring algorithms, accuracy of the segmentation is further improved.

Key words: shape clusters, co-segmentation, weakly supervised learning, local convolution, energy function

摘要：

随着三维采集技术的快速发展,点云数据逐渐成为表示三维模型的基本数据格式之一,它可以保留模型的更多三维空间几何信息。但在三维点云模型分割研究中,大多深度学习网络架构依赖于高质量标注的数据,导致训练成本高昂。因此,针对利用带少量标注点的训练样本实现三维模型簇协同分割的问题,提出一种基于弱监督学习的三维点云模型簇协同一致分割方法。首先,通过K近邻算法建立点之间的局部邻域图;然后,利用局部卷积方法提取点云模型的部件特征并构建相似部件矩阵;最后,通过能量函数反向传播优化网络权值,获得模型簇的一致性分割结果。实验结果表明,该算法在公开数据集ShapeNet Parts上的分割准确率达到85.0%。与现有的有监督算法相比,该算法在训练样本标签数减少至10%的情况下依然能够取得与有监督学习方法接近甚至更好的分割结果,并且与目前主流的弱监督算法相比,分割准确率得到进一步提升。

关键词: 模型簇, 协同分割, 弱监督学习, 局部卷积, 能量函数

CLC Number:

TP391

YANG Jun, LEI Xiwen. Co-segmentation of 3D Point Cloud Shape Clusters Based on Weakly Supervised Learning[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2121-2131.

杨军, 雷喜文. 弱监督学习下的三维点云模型簇协同分割[J]. 计算机科学与探索, 2022, 16(9): 2121-2131.

Figures/Tables 9

References 28

[1]	ZHANG J, ZHENG J, WU C, et al. Variational mesh decom-position[J]. ACM Transactions on Graphics, 2012, 31(3): 1-14.
[2]	LEV J, LIM J H, OUARTI N. Principal curvature of point cloud for 3D shape recognition[C]// Proceedings of the 2017 IEEE International Conference on Image Processing, Beijing, Sep 17-20, 2017. Piscataway: IEEE, 2017: 610-614.
[3]	LI Y, BU R, SUN M, et al. PointCNN: convolution on x-transformed points[C]// Advances in Neural Information Processing Systems 31, Montreal, Dec 3-8, 2018: 820-830.
[4]	QI C R, SU H, MO K, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pat-tern Recognition, Honolulu, Jul 21-27, 2017. Washington: IEEE Computer Society, 2017: 77-85.
[5]	党吉圣, 杨军. 多特征融合的三维模型识别与分割[J]. 西安电子科技大学学报, 2020, 47(4): 149-157.
	DANG J S, YANG J. 3D model recognition and segmen-tation based on multi-feature fusion[J]. Journal of Xidian University, 2020, 47(4): 149-157.
[6]	KALOGERAKIS E, AVERKIOU M, MAJI S, et al. 3D shape segmentation with projective convolutional networks[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 6630-6639.
[7]	HU R Z, FAN L B, LIU L G. Co-segmentation of 3D shapes via subspace clustering[J]. Computer Graphics Forum, 2012, 31(5): 1703-1713. DOI URL
[8]	LAI Y K, HU S M, MARTIN R R, et al. Fast mesh segmen-tation using random walks[C]// Proceedings of the 2008 ACM Symposium on Solid and Physical Modeling, Stony Brook, Jun 2-4, 2008. New York: ACM, 2008: 183-191.
[9]	BENJAMIN W, POLK A W, VISHWANATHAN S V N, et al. Heat walk: robust salient segmentation of non-rigid shapes[J]. Computer Graphics Forum, 2011, 30(7): 2097-2106. DOI URL
[10]	SHLAFMAN S, TAL A, KATZ S. Metamorphosis of poly-hedral surfaces using decomposition[J]. Computer Graphics Forum, 2010, 21(3): 219-228. DOI URL
[11]	LIU R, ZHANG H. Segmentation of 3D meshes through spectral clustering[C]// Proceedings of the 12th Pacific Con-ference on Computer Graphics and Application, Seoul, Oct 6-8, 2004. Washington: IEEE Computer Society, 2004: 298-305.
[12]	HUANG H, KALOGERAKIS E, CHAUDHURI S, et al. Learning local shape descriptors from part correspondences with multi-view convolutional networks[J]. ACM Transac-tions on Graphics, 2017, 37(1).
[13]	杨军, 王顺, 周鹏. 基于深度体素卷积神经网络的三维模型识别分类[J]. 光学学报, 2019, 39(4): 306-316.
	YANG J, WANG S, ZHOU P. Recognition and classifica-tion for three-dimensional model based on deep voxel convo-lution neural network[J]. Acta Optica Sinica, 2019, 39(4): 306-316.
[14]	WANG Y, ASAFI S, KAICK O V, et al. Active co-analysis of a set of shapes[J]. ACM Transactions on Graphics, 2012, 31(6): 157.
[15]	WU Z, SHOU R, WANG Y, et al. Interactive shape co-seg-mentation via label propagation[J]. Computers & Graphics, 2014, 38(2): 248-254. DOI URL
[16]	GOLOVINSKIY A, FUNKHOUSER T. Consistent segmen-tation of 3D models[J]. Computers & Graphics, 2009, 33(3): 262-269. DOI URL
[17]	SIDI O, KAICK O V, KLEIMAN Y, et al. Unsupervised co-segmentation of a set of shapes via descriptor space spectral clustering[J]. ACM Transactions on Graphics, 2011, 30(6).
[18]	SHU Z, QI C, XIN S, et al. Unsupervised 3D shape segmen-tation and co-segmentation via deep learning[J]. Computer Aided Geometric Design, 2016, 43(3): 39-52. DOI URL
[19]	KALOGERAKIS E, HERTZMANN A, SINGH K. Learning 3D mesh segmentation and labeling[J]. ACM Transactions on Graphics, 2010, 29(4): 102.
[20]	GUO K, ZOU D, CHEN X. 3D mesh labeling via deep con-volutional neural networks[J]. ACM Transactions on Graphics, 2015, 35(1): 3.
[21]	QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[C]// Advan-ces in Neural Information Processing Systems 30, Long Beach, Dec 4-9, 2017: 5099-5108.
[22]	ZHAO Y H, BIRDAL T, DENG H W, et al. 3D point capsule networks[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1009-1018.
[23]	CHEN Z Q, YIN K X, FISHER M, et al. BAE-NET: bran-ched autoencoder for shape co-segmentation[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27 - Nov 2, 2019. Piscataway: IEEE, 2019: 8489-8498.
[24]	WEI J C, LIN G S, YAP K H, et al. Multi-path region mining for weakly supervised 3D semantic segmentation on point clouds[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 4384-4393.
[25]	ZHU C Y, XU K, CHAUDHURI S, et al. AdaCoSeg: adap-tive shape co-segmentation with group consistency loss[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Pis-cataway: IEEE, 2020: 8540-8549.
[26]	WANG Y, SUN Y, LIU Z, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 146.
[27]	YI L, KIM V G, CEYLAN D, et al. A scalable active frame-work for region annotation in 3D shape collections[J]. ACM Transactions on Graphics, 2016, 35(6): 210.
[28]	KLOKOV R, LEMPITSKY V. Escape from cells: deep Kd-networks for the recognition of 3D point cloud models[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 863-872.

Algorithms	mIoU	pIoU
PointNet^[4]	80.4	83.6
PointNet++^[21]	81.9	85.1
DGCNN^[26]	82.3	85.1
KD-Net^[28]	77.4	82.3
Capsule-Net^[22]	80.9	84.9
BAE-NET^[23]	79.9	84.6
Our algorithm (1 point)	74.4	75.5
Our algorithm (10% points)	81.6	85.0

Algorithms	mIoU	pIoU
PointNet^[4]	80.4	83.6
PointNet++^[21]	81.9	85.1
DGCNN^[26]	82.3	85.1
KD-Net^[28]	77.4	82.3
Capsule-Net^[22]	80.9	84.9
BAE-NET^[23]	79.9	84.6
Our algorithm (1 point)	74.4	75.5
Our algorithm (10% points)	81.6	85.0

Learning method	Algorithms	Airplane (2 690)	Bag (76)	Cap (55)	Car (898)	Chair (3 758)	Earphone (69)	Guitar (787)	Knife (392)
s	PointNet^[4]	83.2	80.4	86.1	76.6	89.5	70.8	91.2	85.3
s	PointNet++^[21]	82.4	79.0	87.7	77.3	90.8	71.8	91.0	85.9
s	DGCNN^[26]	84.2	83.7	84.4	77.1	90.9	78.5	91.5	87.3
s	KD-Net^[28]	80.1	74.6	74.3	70.3	88.6	73.5	90.2	87.2
w	Capsule-Net^[22]	82.8	77.8	88.0	77.3	89.6	73.5	90.7	83.9
w	BAE-NET^[23]	81.2	72.7	79.9	76.5	88.3	70.4	90.0	80.5
w	Our algorithm (1 point)	75.6	74.8	79.2	66.5	87.3	63.3	89.4	84.2
w	Our algorithm (10% points)	83.7	82.6	80.6	77.8	89.8	77.3	90.9	87.6
Learning method	Algorithms	Lamp (1 547)	Laptop (451)	Motorbike (202)	Mug (184)	Pistol (283)	Rocket (66)	Skateboard (152)	Table (5 271)
s	PointNet^[4]	80.4	95.3	64.5	91.8	81.3	61.3	72.8	80.4
s	PointNet++^[21]	83.7	95.3	71.6	94.1	81.3	58.7	76.4	82.6
s	DGCNN^[26]	82.9	96.0	67.8	93.3	82.6	59.7	75.5	82.0
s	KD-Net^[28]	81.0	94.9	57.4	86.7	78.1	51.8	69.9	80.3
w	Capsule-Net^[22]	82.8	94.8	69.1	93.2	80.9	53.1	72.9	83.0
w	BAE-NET^[23]	76.1	95.1	60.5	89.8	80.8	57.1	78.3	88.1
w	Our algorithm (1 point)	78.7	94.5	49.7	90.3	76.7	46.5	71.3	62.6
w	Our algorithm (10% points)	82.5	95.8	64.7	93.5	79.8	61.9	74.9	82.9

Learning method	Algorithms	Airplane (2 690)	Bag (76)	Cap (55)	Car (898)	Chair (3 758)	Earphone (69)	Guitar (787)	Knife (392)
s	PointNet^[4]	83.2	80.4	86.1	76.6	89.5	70.8	91.2	85.3
s	PointNet++^[21]	82.4	79.0	87.7	77.3	90.8	71.8	91.0	85.9
s	DGCNN^[26]	84.2	83.7	84.4	77.1	90.9	78.5	91.5	87.3
s	KD-Net^[28]	80.1	74.6	74.3	70.3	88.6	73.5	90.2	87.2
w	Capsule-Net^[22]	82.8	77.8	88.0	77.3	89.6	73.5	90.7	83.9
w	BAE-NET^[23]	81.2	72.7	79.9	76.5	88.3	70.4	90.0	80.5
w	Our algorithm (1 point)	75.6	74.8	79.2	66.5	87.3	63.3	89.4	84.2
w	Our algorithm (10% points)	83.7	82.6	80.6	77.8	89.8	77.3	90.9	87.6
Learning method	Algorithms	Lamp (1 547)	Laptop (451)	Motorbike (202)	Mug (184)	Pistol (283)	Rocket (66)	Skateboard (152)	Table (5 271)
s	PointNet^[4]	80.4	95.3	64.5	91.8	81.3	61.3	72.8	80.4
s	PointNet++^[21]	83.7	95.3	71.6	94.1	81.3	58.7	76.4	82.6
s	DGCNN^[26]	82.9	96.0	67.8	93.3	82.6	59.7	75.5	82.0
s	KD-Net^[28]	81.0	94.9	57.4	86.7	78.1	51.8	69.9	80.3
w	Capsule-Net^[22]	82.8	94.8	69.1	93.2	80.9	53.1	72.9	83.0
w	BAE-NET^[23]	76.1	95.1	60.5	89.8	80.8	57.1	78.3	88.1
w	Our algorithm (1 point)	78.7	94.5	49.7	90.3	76.7	46.5	71.3	62.6
w	Our algorithm (10% points)	82.5	95.8	64.7	93.5	79.8	61.9	74.9	82.9

Sampling	LocalConv	Energy function	mIoU/%
Replaced with random sampling	√	√	71.2
√		√	56.3
√	√		67.1
√	√	√	81.6

Co-segmentation of 3D Point Cloud Shape Clusters Based on Weakly Supervised Learning

弱监督学习下的三维点云模型簇协同分割

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 28

Related Articles 1

Recommended Articles

Metrics

Label Strategy	mIoU	pIoU
100% Sample + 100%Pts	82.19	85.10
100% Sample + 50%Pts	82.07	85.02
100% Sample + 10%Pts	81.70	85.00