改进双分支胶囊网络的高光谱图像分类

doi:10.3778/j.issn.1673-9418.2102073

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (10): 2405-2414.DOI: 10.3778/j.issn.1673-9418.2102073

• 图形图像 • 上一篇

改进双分支胶囊网络的高光谱图像分类

张海涛, 柴思敏⁺()

辽宁工程技术大学软件学院,辽宁葫芦岛 125105

收稿日期:2021-03-01 修回日期:2021-06-07 出版日期:2022-10-01 发布日期:2021-06-23
通讯作者: + E-mail: 1149278938@qq.com
作者简介:张海涛（1974—）,男,博士,教授,硕士生导师,CCF会员,主要研究方向为机器学习、图形图像处理等。
柴思敏（1996—）,男,硕士研究生,主要研究方向为机器学习、图形图像处理等。
基金资助:
国家部委预研基金;辽宁省自然科学基金面上项目(20170540426)

Improved Two-Branch Capsule Network for Hyperspectral Image Classification

ZHANG Haitao, CHAI Simin⁺()

Software College, Liaoning Technical University, Huludao, Liaoning 125105, China

Received:2021-03-01 Revised:2021-06-07 Online:2022-10-01 Published:2021-06-23
About author:ZHANG Haitao, born in 1974, Ph.D., profes-sor, M.S. supervisor, member of CCF. His resea-rch interests include machine learning, graphics and image processing, etc.
CHAI Simin, born in 1996, M.S. candidate. His research interests include machine learning, graphics and image processing, etc.
Supported by:
Pre-research Foundation of National Ministries and Commissions;Natural Science Foundation of Liaoning Province (General Program)(20170540426)

摘要/Abstract

摘要：

基于双分支的胶囊网络分类方法在两个通道分别提取光谱信息和空间信息,既保留了双分支卷积神经网络的特征提取方式,又提高了分类精度。但由于高光谱图像（HSI）通常由几百个通道组成,在训练胶囊网络时,动态路由过程产生了大量的训练参数。为此提出1D和2D约束窗口分别减少来自两个提取通道的胶囊数量。它以胶囊向量组为计算单位进行卷积运算,来减少胶囊网络的参数量和计算复杂度。基于该降参优化方法提出一个新的双分支胶囊神经网络（DuB-ConvCapsNet-MRF）,并将其应用在高光谱图像分类任务中。此外,为进一步提高分类性能,引入马尔可夫随机场（MRF）对空间区域进行平滑后处理,获得最终输出。对两个代表性高光谱图像数据集进行消融实验并与现有的6个分类方法进行比较,结果表明,DuB-ConvCapsNet-MRF在分类精度上都优于其他方法,并且有效降低了胶囊网络的训练代价。

关键词: 遥感, 高光谱图像分类, 胶囊神经网络, 约束窗口, 马尔可夫随机场（MRF）

Abstract:

The method based on the dual-channel capsule network extracts spectral information and spatial informa-tion separately in two channels, which not only retains the feature extraction method of the dual-channel convolu-tional neural network, but also improves the classification accuracy. However, when researchers train the capsule network, the dynamic routing process generates a large number of training parameters because the hyperspectral image (HSI) usually consists of hundreds of channels. To address this limitation, 1D and 2D constraint windows are proposed to reduce the number of capsules from two extraction channels. It uses the capsule vector group as the calculation unit to perform convolution operations and reduce the amount of parameters and computational com-plexity of the capsule network. Based on this parameter reduction optimization method, a new dual-branch capsule neural network (DuB-ConvCapsNet-MRF) is proposed and applied to the task of hyperspectral image classifica-tion. In addition, in order to further improve the classification accuracy, Markov random field (MRF) is introduced to smooth the spatial region and the final output is got. The results of performing ablation experiments on two repre-sentative hyperspectral image datasets and comparing the proposed method with six existing classification methods show that DuB-ConvCapsNet-MRF is superior to other methods in classification performance, and effectively re-duces the cost of training capsule network.

Key words: remote sensing, hyperspectral image classification, capsule neural network, constraint window, Markov random field (MRF)

中图分类号:

TP751.1

张海涛, 柴思敏. 改进双分支胶囊网络的高光谱图像分类[J]. 计算机科学与探索, 2022, 16(10): 2405-2414.

ZHANG Haitao, CHAI Simin. Improved Two-Branch Capsule Network for Hyperspectral Image Classification[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(10): 2405-2414.

图/表 16

图1 DuB-ConvCapsNet-MRF网络架构

Fig.1 Network architecture of DuB-ConvCapsNet-MRF

图2 动态路由的执行过程

Fig.2 Execution process of dynamic routing

图3 所提出的降低参数方法2D-ConvCaps

Fig.3 Proposed parameter reduction method, 2D-ConvCaps

图4 所提出的降低参数方法1D-ConvCaps

Fig.4 Proposed parameter reduction method, 1D-ConvCaps

图5 Indian Pines数据集

Fig.5 Indian Pines datasets

图6 University of Pavia数据集

Fig.6 University of Pavia datasets

表1 用于训练所提出的DuB-ConvCapsNet-MRF的超参数

Table 1 Hyper-parameters used for training proposed DuB-ConvCapsNet-MRF

Hyper-parameter	Optimized value
Optimizer	Adam
Number of epochs	100
Dropout	0.2
Learning rate	0.01
Learning rate decay	0.9
PCA	5
$λ$ (in Eq. (7))	0.5
$m +$ (in Eq. (7))	0.9
$m -$ (in Eq. (7))	0.1

表1 用于训练所提出的DuB-ConvCapsNet-MRF的超参数

Table 1 Hyper-parameters used for training proposed DuB-ConvCapsNet-MRF

Hyper-parameter	Optimized value
Optimizer	Adam
Number of epochs	100
Dropout	0.2
Learning rate	0.01
Learning rate decay	0.9
PCA	5
$λ$ (in Eq. (7))	0.5
$m +$ (in Eq. (7))	0.9
$m -$ (in Eq. (7))	0.1

图7 PCA降维保留的通道数对分类性能的影响

Fig.7 Impact of the number of channels reserved by PCA on classification performance

表2 不同方法在IP数据集上的分类结果

Table 2 Classification results of different

Class	SVM	2D-CNN	3D- CNN	DCD-CNN	CNN-MRF	Caps-MRF	Proposed
1	77.88	95.67	98.16	98.45	90.08	89.92	97.72
2	0	94.31	97.86	98.77	96.91	98.49	99.31
3	25.16	91.76	99.85	96.95	97.86	97.62	96.26
4	68.17	81.38	98.13	98.13	96.81	98.84	99.46
5	46.54	98.46	98.64	100.00	97.54	99.91	100.00
6	80.77	99.42	100.00	98.03	99.35	98.87	98.13
7	35.97	96.47	99.46	92.24	92.00	96.56	97.51
8	97.07	100.00	99.51	100.00	99.46	100.00	98.16
9	45.00	62.46	60.00	60.00	68.48	71.79	73.26
10	71.70	95.78	97.89	99.67	98.60	99.78	100.00
11	86.92	97.32	98.67	98.92	97.91	99.17	99.84
12	61.73	93.78	100.00	99.41	98.46	98.82	97.94
13	91.85	97.89	98.87	100.00	100.00	100.00	100.00
14	96.40	99.67	99.46	97.21	95.84	98.26	98.82
15	45.53	87.61	99.21	99.16	100.00	99.16	98.63
16	82.81	88.63	96.16	89.71	85.61	99.25	92.17
OA	77.63	95.13	98.45	98.37	97.15	98.53	98.74
AA	63.34	92.54	96.37	95.36	94.68	96.65	96.70
Kappa	84.21	94.91	98.71	98.23	98.31	98.47	98.93
训练时间/s	12	2 426	33 413	3 120	5 124	21 670	13 268

表3 不同情况下的模型在IP数据集的分类准确性

Table 3 Classification accuracy of models in different situations on IP dataset

MRF	2D-ConvCaps/ 1D-ConvCaps	PCA	OA/%	Kappa/%
√			92.13	90.76
√	√		93.79	92.82
	√	√	97.21	96.48
√	√	√	98.74	98.93

图8 IP数据集中不同比率训练样本的分类准确性

Fig.8 Classification accuracy of different ratios of training samples in IP dataset

图9 不同方法在IP数据集的分类图

Fig.9 Classification maps of different methods on IP dataset

表4 不同情况下的模型在UP数据集的分类准确性

Table 4 Classification accuracy of models in different situations on UP dataset

MRF	2D-ConvCaps/ 1D-ConvCaps	PCA	OA/%	Kappa/%
√			93.94	91.90
√	√		95.76	93.58
	√	√	98.13	97.16
√	√	√	99.61	99.34

图10 UP数据集中不同比率训练样本的分类准确性

Fig.10 Classification accuracy of different ratios of training samples in UP dataset

表5 不同方法在UP数据集上的分类结果单位：%

Table 5 Classification results of different methods on UP dataset

class	SVM	2D-CNN	3D-CNN	DCD-CNN	CNN-MRF	Caps-MRF	Proposed
1	75.45	87.61	99.65	98.75	97.41	99.97	99.46
2	92.41	88.57	99.17	99.16	97.25	98.46	99.43
3	96.23	89.42	94.76	100.00	90.64	98.72	100.00
4	95.76	90.57	99.29	98.13	96.71	99.67	99.71
5	98.01	90.64	100.00	99.87	99.18	99.16	99.13
6	93.98	70.67	98.74	100.00	98.34	99.43	98.87
7	82.71	88.96	96.96	100.00	98.74	99.56	100.00
8	64.64	89.79	99.26	99.34	90.74	99.81	99.67
9	97.81	91.13	99.16	99.16	99.28	98.46	99.38
OA	75.69	87.46	98.91	99.57	96.16	99.24	99.61
AA	88.56	87.48	98.55	99.38	96.48	99.25	99.52
Kappa	70.04	85.17	99.22	99.26	95.33	97.98	99.34
训练时间/s	46	7 261	100 496	9 346	14 962	63 570	39 715

表6 输入和特征提取的不同向量组个数带来的参数量

Table 6 Amount of parameters brought by different number of vector groups for input and feature extraction

$a 1$	$a 2$
$a 1$	4	8	16	32	64	未降参
64	354 816	709 632	1 419 264	2 838 528	5 677 056	4 956 160
32	281 088	562 176	1 124 352	2 248 704	4 497 408	2 478 080
16	244 224	488 448	976 896	1 953 792	3 907 584	1 239 040

表6 输入和特征提取的不同向量组个数带来的参数量

Table 6 Amount of parameters brought by different number of vector groups for input and feature extraction

$a 1$	$a 2$
$a 1$	4	8	16	32	64	未降参
64	354 816	709 632	1 419 264	2 838 528	5 677 056	4 956 160
32	281 088	562 176	1 124 352	2 248 704	4 497 408	2 478 080
16	244 224	488 448	976 896	1 953 792	3 907 584	1 239 040

参考文献 27

[1]	CAO X H, LI R J, GE Y M, et al. Densely connected deep random forest for hyperspectral imagery classification[J]. Inter-national Journal of Remote Sensing, 2019, 40(9): 3606-3622.
[2]	LANTHIER Y, BANNARI A, HABOUDANE D, et al. Hy-perspectral data segmentation and classification in preci-sion agriculture: a multi-scale analysis[C]// Proceedings of the 2008 IEEE International Geoscience & Remote Sensing Symposium, Boston, Jul 8-11, 2008. Piscataway: IEEE, 2008: 585-588.
[3]	GHAMISI P, YOKOYA N, LI J, et al. Advances in hyper-spectral image and signal processing: a comprehensive over-view of the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(4): 37-78. DOI URL
[4]	WANG Q, LIU S T, CHANUSSOT J, et al. Scene classifi-cation with recurrent attention of VHR remote sensing ima-ges[J]. IEEE Transactions on Geoscience and Remote Sen-sing, 2018, 57(2): 1155-1167.
[5]	LANDGREBE D. Hyperspectral image data analysis[J]. IEEE Signal Processing Magazine, 2002, 19(1): 17-28.
[6]	FAUVEL M, TARABALKA Y, BENEDIKTSSON J A, et al. Advances in spectral-spatial classification of hyperspectral images[J]. Proceedings of the IEEE, 2012, 101(3): 652-675. DOI URL
[7]	HAM J, CHEN Y, CRAWFORD M M, et al. Investigation of the random forest framework for classification of hyper-spectral data[J]. IEE20E Transactions on Geoscience and Remote Sensing, 2005, 43(3): 492-501.
[8]	CHEN Y, NASRABADI N M, TRAN T D. Hyperspectral image classification using dictionary-based sparse represen-tation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(10): 3973-3985. DOI URL
[9]	HAUT J M, PAOLETTI M, PLAZA J, et al. Cloud implemen-tation of the K-means algorithm for hyperspectral image analysis[J]. The Journal of Supercomputing, 2017, 73(1): 514-529. DOI URL
[10]	LIU W B, WANG Z D, LIU X H, et al. A survey of deep neural network architectures and their applications[J]. Neuro-computing, 2017, 234: 11-26.
[11]	CHEN Y S, LIN Z H, ZHAO X, et al. Deep learning-based classification of hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6): 2094-2107. DOI URL
[12]	CHEN Y S, ZHAO X, JIA X P. Spectral-spatial classifica-tion of hyperspectral data based on deep belief network[J]. IEEE Journal of Selected Topics in Applied Earth Obser-vations and Remote Sensing, 2015, 8(6): 2381-2392.
[13]	万亚玲, 钟锡武, 刘慧, 等. 卷积神经网络在高光谱图像分类中的应用综述[J]. 计算机工程与应用, 2021, 57(4): 1-10. DOI
	WAN Y L, ZHONG X W, LIU H, et al. Survey of applica-tion of convolutional neural network in classification of hyperspectral images[J]. Computer Engineering and Appli-cations, 2021, 57(4): 1-10.
[14]	MAKANTASIS K, KARANTZALOS K, DOULAMIS A D, et al. Deep supervised learning for hyperspectral data classi-fication through convolutional neural networks[C]// Procee-dings of the 2015 IEEE International Geoscience and Re-mote Sensing Symposium, Milan, Jul 26-31, 2015. Piscata-way: IEEE, 2015: 4959-4962.
[15]	HAMIDA A B, BENOIT A, LAMBERT P, et al. 3-D deep learning approach for remote sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(8): 4420-4434. DOI URL
[16]	佘海龙, 解山娟, 邹静洁. 标准分数降维的3D-CNN高光谱遥感图像分类[J]. 计算机工程与应用, 2021, 57(4): 169-175. DOI
	SHE H L, XIE S J, ZOU J J. 3D-CNN with standard score dimensionality reduction for hyperspectral remote sensing images classification[J]. Computer Engineering and Appli-cations, 2021, 57(4): 169-175.
[17]	XU X D, LI W, RAN Q, et al. Multisource remote sensing data classification based on convolutional neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(2): 937-949. DOI URL
[18]	刘启超, 肖亮, 刘芳, 等. SSCDenseNet: 一种空-谱卷积稠密网络的高光谱图像分类算法[J]. 电子学报, 2020, 48(4): 751-762. DOI
	LIU Q C, XIAO L, LIU F, et al. SSCDenseNet: a spectral-spatial convolutional dense network for hyperspectral image classification[J]. Acta Electronica Sinica, 2020, 48(4): 751-762.
[19]	SABOUR S, FROSST N, HINTON G E. Dynamic routing between capsules[J]. arXiv:1710.09829, 2017.
[20]	JIANG X F, LIU W B, ZHANG Y, et al. Spectral-spatial hy-perspectral image classification using dual-channel capsule networks[J]. IEEE Geoscience and Remote Sensing Let-ters, 2021, 18(6): 1094-1098.
[21]	JIANG X F, ZHANG Y, LIU W B, et al. Hyperspectral ima-ge classification with CapsNet and Markov random fields[J]. IEEE Access, 2020, 8: 191956-191968. DOI URL
[22]	ZHANG H T, MENG L G, WEI X, et al. 1D-convolutional capsule network for hyperspectral image classification[J]. arXiv:1903.09834, 2019.
[23]	TARABALKA Y, FAUVEL M, CHANUSSOT J, et al. SVM- and MRF-based method for accurate classification of hy-perspectral images[J]. IEEE Geoscience and Remote Sen-sing Letters, 2010, 7(4): 736-740.
[24]	YUE J, ZHAO W Z, MAO S J, et al. Spectral-spatial clas-sification of hyperspectral images using deep convolutio-nal neural networks[J]. Remote Sensing Letters, 2015, 6(6): 468-477. DOI URL
[25]	LI Y, ZHANG H K, SHEN Q. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network[J]. Remote Sensing, 2017, 9(1): 67. DOI URL
[26]	胡丽, 单锐, 王芳, 等. 基于双通道空洞卷积神经网络的高光谱图像分类[J]. 激光与光电子学进展, 2020, 57(12): 348-354.
	HU L, SHAN R, WANG F, et al. Hyperspectral image clas-sification based on dual-channel dilated convolution neural network[J]. Laser & Optoelectronics Progress, 2020, 57(12): 348-354.
[27]	CAO X Y, ZHOU F, XU L, et al. Hyperspectral image classi-fication with Markov random fields and a convolutional neural network[J]. IEEE Transactions on Image Processing, 2018, 27(5): 2354-2367. DOI URL

改进双分支胶囊网络的高光谱图像分类

Improved Two-Branch Capsule Network for Hyperspectral Image Classification

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献 27

相关文章 6

编辑推荐

Metrics

[1]	王振华，何婉雯，孙婧琦，曲念毅，黄冬梅. 改进水平集模型的海岛边界快速分割方法研究[J]. 计算机科学与探索, 2019, 13(7): 1206-1216.
[2]	祖宝开，夏克文，牛文佳，姜晓庆. 分块低秩图的遥感影像半监督分类应用[J]. 计算机科学与探索, 2019, 13(7): 1217-1226.
[3]	季艳，鲁克文，张英慧. 海量遥感数据分布式集群化存储技术研究[J]. 计算机科学与探索, 2017, 11(9): 1398-1404.
[4]	陈艺虾，孙权森，徐焕宇，耿蕾蕾. SURF算法和RANSAC算法相结合的遥感图像匹配方法[J]. 计算机科学与探索, 2012, 6(9): 822-828.
[5]	高辉, 张茂军, 徐玮. 遥感图像辅助的折反射全向图三维重构[J]. 计算机科学与探索, 2011, 5(2): 147-154.
[6]	吴昊1 , 李士进1+ , 林林2 , 万定生1 . 多策略结合的高光谱图像波段选择新方法*[J]. 计算机科学与探索, 2010, 4(5): 464-472.