Isomorphic Graph Classification Model Based on Reconstruction Error

doi:10.3778/j.issn.1673-9418.2009049

Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (1): 185-193.DOI: 10.3778/j.issn.1673-9418.2009049

• Artificial Intelligence • Previous Articles Next Articles

Isomorphic Graph Classification Model Based on Reconstruction Error

JIANG Guangfeng, HU Pengcheng, YE Hua, YANG Yanlan⁺()

School of Automation, Southeast University, Nanjing 210096, China

Received:2020-09-17 Revised:2020-11-11 Online:2022-01-01 Published:2020-12-03
About author:JIANG Guangfeng, born in 1995, M.S. candidate. His research interests include pattern recognition and intelligent system, computer vision and video understanding.
HU Pengcheng, born in 1996, M.S. candidate. His research interests include pattern recognition and intelligent system, recommendation system and network information security.
YE Hua, born in 1961, Ph.D., professor. His research interests include intelligence control, pattern recognition, computer application, intelligence building, intelligence robot, fault diagnosis, etc.
YANG Yanlan, born in 1981, Ph.D., lecturer. Her research interests include pattern recognition and intelligent system, computer network communication, large database modeling and optimization, etc.
Supported by:
Fundamental Research Funds for the Central Universities of China(2242020K40244)

基于重构误差的同构图分类模型

蒋光峰, 胡鹏程, 叶桦, 仰燕兰⁺()

东南大学自动化学院,南京 210096

通讯作者: + E-mail: yyl@seu.edu.cn
作者简介:蒋光峰（1995—）,男,安徽马鞍山人,硕士研究生,主要研究方向为模式识别与智能系统、计算机视觉、视频理解。
胡鹏程（1996—）,男,江苏宿迁人,硕士研究生,主要研究方向为模式识别与智能系统、推荐系统、网络信息安全。
叶桦（1961—）,男,江苏南京人,博士,教授,主要研究方向为智能控制、模式识别、计算机应用、智能大厦、智能机器人、故障诊断等。
仰燕兰（1981—）,女,江苏苏州人,博士,讲师,主要研究方向为模式识别与智能系统、计算机网络通信、大型数据库建模与优化等。
基金资助:
中央高校基本科研业务费专项资金(2242020K40244)

Abstract

Abstract:

At present, the application of deep learning method in graph classification model focuses on the migration of convolutional neural network to graph data field, including redefinition of convolutional layer and pooling layer. Generalization of convolution operation to graph data is an effective method. However, both the convolutional layer and the global pooling layer have great room for improvement, especially in the extraction of network topology information. A new isomorphism classification model based on reconstruction error is proposed. On the one hand, WaveGIC is used to improve the ability of extracting topology information. On the other hand, multi-attention mechanism is used to represent the whole picture, which enables the model to pay attention to the information of key nodes. Due to the network deepening process, the characteristic expression of local topological structure is less and less obvious. Based on the classification loss, the reconstruction error loss is added to make the classifier consider the node characteristics and topology structure of the graph at the same time. Experimental results on the benchmark data set show that the proposed method has high accuracy of graph classification.

Key words: graph neural network (GNN), graph classification, reconstruction error, attention mechanism

摘要：

目前深度学习方法应用于图分类模型的重点集中在将卷积神经网络迁移到图数据领域,包括重定义卷积层和池化层。卷积操作泛化到图数据上是有效的方法,但无论是卷积还是池化都存在较大的改进空间,尤其是在提取网络拓扑结构信息方面。提出一种基于重构误差的同构图分类模型,一方面利用改进的同构图卷积网络WaveGIC增强提取拓扑结构信息能力;另一方面利用多重注意力机制表征全图,使得模型能够关注关键节点信息。由于网络加深过程,局部拓扑结构的特征表达越来越不明显。在分类损失基础上添加重构误差损失,使分类器同时考虑图的节点特征和拓扑结构。在基准数据集上的实验结果表明,提出的方法具有较高的图分类准确度。

关键词: 图神经网络（GNN）, 图分类, 重构误差, 注意力机制

CLC Number:

TP183

JIANG Guangfeng, HU Pengcheng, YE Hua, YANG Yanlan. Isomorphic Graph Classification Model Based on Reconstruction Error[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(1): 185-193.

蒋光峰, 胡鹏程, 叶桦, 仰燕兰. 基于重构误差的同构图分类模型[J]. 计算机科学与探索, 2022, 16(1): 185-193.

Figures/Tables 10

References 33

[1]	BORGWARDT K M, ONG C S, ScHÖNAUER S, et al. Protein function prediction via graph kernels[J]. Bioinformatics, 2005, 21(S1):i47-i56. DOI URL
[2]	DUVENAUD D K, MACLAURIN D, AGUILERA-IPARRAGUIRRE J, et al. Convolutional networks on graphs for learning molecular fingerprints[C]// Proceedings of the Annual Conference on Neural Information Processing Systems, Montreal, Dec 7-12, 2015. Red Hook: Curran Associates, 2015: 2224-2232.
[3]	BACKSTROM L, LESKOVEC J. Supervised random walks: predicting and recommending links in social networks[C]// Proceedings of the 2011 International Conference on Web Search and Data Mining, Hong Kong, China, Feb 9-12, 2011. New York: ACM, 2011: 635-644.
[4]	BAO J, HE T F, RUAN S J, et al. Planning bike lanes based on sharing-bikes trajectories[C]// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, Aug 13-17, 2017. New York: ACM, 2017: 1377-1386.
[5]	CHAU D H P, NACHENBERG C, WILHELM J, et al. Polonium: tera-scale graph mining and inference for malware detec-tion[C]// Proceedings of the 2011 SIAM International Con-ference on Data Mining, Hilton Phoenix East, Apr 28-30, 2011. Philadelphia: SIAM, 2011: 131-142.
[6]	LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324. DOI URL
[7]	BENGIO Y, BOULANGER-LEWANDOWSKI N, PASCANU R. Advances in optimizing recurrent networks[C]// Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, May 26-31, 2013. Piscataway: IEEE, 2013: 8624-8628.
[8]	BRUNA J, ZAREMBA W, SZLAM A, et al. Spectral networks and locally connected networks on graphs[J]. arXiv:1312.6203, 2013
[9]	DEFFERRARD M, BRESSON X, VANDERGHEYNST P. Convolutional neural networks on graphs with fast localized spectral filtering[C]// Proceedings of the Annual Conference on Neural Information Processing Systems, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates, 2016: 3844-3852.
[10]	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[J]. arXiv:1609.02907, 2016.
[11]	HAMILTON W L, YING Z T, LESKOVEC J. Inductive representation learning on large graphs[C]// Proceedings of the Annual Conference on Neural Information Processing Systems, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 1024-1034.
[12]	VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph attention networks[J]. arXiv:1710.10903, 2017.
[13]	RHEE S, SEO S, KIM S. Hybrid approach of relation network and localized graph convolutional filtering for breast cancer subtype classification[J]. arXiv:1711.05859, 2017.
[14]	ZHANG M H, CUI Z C, NEUMANN M, et al. An end-to-end deep learning architecture for graph classification[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2018: 4438-4445.
[15]	YING Z, YOU J, MORRIS C, et al. Hierarchical graph repre-sentation learning with differentiable pooling[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence, the 30th Innovative Applications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 4800-4810.
[16]	LEE J, LEE I, KANG J. Self-attention graph pooling[J]. arXiv:1904.08082, 2019.
[17]	GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Neural message passing for quantum chemistry[C]// Proceedings of the 34th International Conference on Machine Learning, Sydney, Aug 6-11, 2017: 1263-1272.
[18]	ZHOU J, CUI G Q, ZHANG Z Y, et al. Graph neural networks: a review of methods and applications[J]. arXiv:1812.08434, 2018.
[19]	WU Z H, PAN S R, CHEN F W, et al. A comprehensive survey on graph neural networks[J]. arXiv:1901.00596, 2019.
[20]	XU K, HU W H, LESKOVEC J, et al. How powerful are graph neural networks[J]. arXiv:1810.00826, 2018.
[21]	VINYALS O, BENGIO S, KUDLUR M. Order matters: sequence to sequence for sets[J]. arXiv:1511.06391, 2015.
[22]	KIPF T N, WELLING M. Variational graph auto-encoders[J]. arXiv:1611.07308, 2016.
[23]	PAN S R, HU R Q, LONG G D, et al. Adversarially regular-ized graph autoencoder for graph embedding[C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Jul 13-19, 2018: 2609-2615.
[24]	OORD A, DIELEMAN S, ZEN H, et al. WaveNet: a generative model for raw audio[J]. arXiv:1609.03499, 2016.
[25]	RIESEN K, BUNKE H. IAM graph database repository for graph based pattern recognition and machine learning[C]// LNCS 5342: Proceedings of the Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition and Structural and Syntactic Pattern Recognition, Orlando, Dec 4-6, 2008. Berlin, Heidelberg: Springer, 2008: 287-297.
[26]	ORSINI F, FRASCONI P, DE RAEDT L. Graph invariant kernels[C]// Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Jul 25-31, 2015. Menlo Park: AAAI, 2015: 3756-3762.
[27]	COSTA F, DE GRAVE K. Fast neighborhood subgraph pairwise distance kernel[C]// Proceedings of the 27th International Conference on Machine Learning, Haifa, Jun 21-24, 2010. Madison: Omnipress, 2010: 255-262.
[28]	WALE N, WATSON I A, KARYPIS G. Comparison of descriptor spaces for chemical compound retrieval and classification[J]. Knowledge and Information Systems, 2008, 14(3):347-375. DOI URL
[29]	DOBSON P D, DOIG A J. Distinguishing enzyme structures from non-enzymes without alignments[J]. Journal of Molecular Biology, 2003, 330(4):771-783. DOI URL
[30]	ShCHUR O, MUMME M, BOJCHEVSKI A, et al. Pitfalls of graph neural network evaluation[J]. arXiv:1811.05868, 2018.
[31]	XU K, LI C T, TIAN Y L, et al. Representation learning on graphs with jumping knowledge networks[J]. arXiv:1806.03536, 2018
[32]	PASZKE A, GROSS S, MASSA F, et al. Pytorch: an imperative style, high-performance deep learning library[C]// Proceedings of the Advances in Neural Information Processing Systems, 2019: 8026-8037.
[33]	WANG M J, YU L F, ZHENG D, et al. Deep graph library: towards efficient and scalable deep learning on graphs[J]. arXiv:1909.01315, 2019.

数据集	样本数	类别数	平均节点数	平均边数	节点特征维度
AIDS	2 000	2	15.69	16.20	4
FRANKENSTEIN	4 337	2	16.90	17.88	780
NCI1	4 110	2	29.87	32.30	0
NCI109	4 127	2	29.68	32.13	0
PROTEINS	1 113	2	39.06	72.82	29

数据集	样本数	类别数	平均节点数	平均边数	节点特征维度
AIDS	2 000	2	15.69	16.20	4
FRANKENSTEIN	4 337	2	16.90	17.88	780
NCI1	4 110	2	29.87	32.30	0
NCI109	4 127	2	29.68	32.13	0
PROTEINS	1 113	2	39.06	72.82	29

模型	图神经网络	读出层	重构误差
GCN	GCN	Sum&MaxPooling	无
Set2Set	GCN	Set2Set	无
SortPool	GCN	SortPool	无
SAGPool	GCN	SAGPool	无
WaveGIC	WaveGIC	Sum&MaxPooling	无
MHAWaveGIC	WaveGIC	MHA Pooling	无
ReWaveGIC	WaveGIC	Sum&MaxPooling	有
RMAWaveGIC	WaveGIC	MHA Pooling	有

模型	图神经网络	读出层	重构误差
GCN	GCN	Sum&MaxPooling	无
Set2Set	GCN	Set2Set	无
SortPool	GCN	SortPool	无
SAGPool	GCN	SAGPool	无
WaveGIC	WaveGIC	Sum&MaxPooling	无
MHAWaveGIC	WaveGIC	MHA Pooling	无
ReWaveGIC	WaveGIC	Sum&MaxPooling	有
RMAWaveGIC	WaveGIC	MHA Pooling	有

Model	Index	AIDS	FRANKENSTEIN	NCI1	NCI109	PROTEINS
GCN	Accuracy/% ROC-AUC	96.80±5.97 0.933 9±0.175 1	64.49±2.74 0.672 9±0.035 2	77.49±2.50 0.835 0±0.019 6	72.86±2.48 0.787 7±0.025 0	63.54±6.54 0.662 5±0.052 3
Set2Set	Accuracy/% ROC-AUC	97.20±1.38 0.948 6±0.053 8	63.94±3.55 0.665 6±0.040 1	75.84±3.09 0.807 7±0.027 4	73.73±3.49 0.780 1±0.031 2	73.13±5.24 0.758 6±0.046 2
SortPool	Accuracy/% ROC-AUC	99.75±0.49 0.995 4±0.007 5	60.99±1.68 0.649 9±0.027 2	77.93±2.77 0.838 6±0.020 8	74.19±3.75 0.795 4±0.037 1	74.75±4.59 0.781 6±0.039 9
SAGPool	Accuracy/% ROC-AUC	96.95±6.03 0.937 7±0.163 6	67.03±2.87 0.713 3±0.025 8	77.01±1.65 0.833 5±0.022 0	75.19±3.13 0.812 1±0.022 1	68.54±12.13 0.705 8±0.111 2
WaveGIC	Accuracy/% ROC-AUC	99.05±0.61 0.986 8±0.017 4	67.44±2.14 0.724 4±0.027 9	75.89±2.56 0.822 8±0.022 7	76.08±2.31 0.816 4±0.020 8	76.91±3.59 0.798 0±0.035 6
MHAWaveGIC	Accuracy/% ROC-AUC	99.25±0.63 0.988 9±0.015 8	69.75±1.83 0.746 2±0.021 9	78.56±2.97 0.834 1±0.0241 6	77.97±2.12 0.818 0±0.020 7	77.27±2.83 0.809 5±0.029 6
ReWaveGIC	Accuracy/% ROC-AUC	98.85±0.82 0.984 7±0.019 9	68.64±1.78 0.725 8±0.024 1	76.52±3.41 0.824 5±0.027 4	75.86±4.09 0.809 5±0.041 7	76.28±5.02 0.798 5±0.048 5
RMAWaveGIC	Accuracy/% ROC-AUC	99.20±0.67 0.990 8±0.013 2	69.43±1.64 0.738 7±0.020 3	79.80±1.99 0.844 8±0.018 3	78.99±2.00 0.837 5±0.020 2	76.63±3.21 0.799 3±0.033 4

Isomorphic Graph Classification Model Based on Reconstruction Error

基于重构误差的同构图分类模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 33

Related Articles 15

Recommended Articles

Metrics

Model	Index	AIDS	FRANKENSTEIN	NCI1	NCI109	PROTEINS
GCN	Accuracy/% ROC-AUC	93.55±9.36 0.836 0±0.256 5	64.49±1.97 0.666 4±0.033 7	77.49±1.78 0.837 3±0.019 7	75.50±2.50 0.814 1±0.025 8	65.40±5.44 0.650 4±0.051 4
Set2Set	Accuracy/% ROC-AUC	99.00±0.71 0.991 9±0.010 9	65.97±2.86 0.698 1±0.037 1	76.55±2.77 0.821 2±0.026 2	75.69±2.01 0.798 7±0.024 9	73.23±3.79 0.765 1±0.042 7
SortPool	Accuracy/% ROC-AUC	99.75±0.35 0.995 5±0.006 8	65.69±2.72 0.701 7±0.019 7	78.23±2.83 0.843 1±0.023 0	75.43±4.19 0.819 1±0.049 1	71.52±4.06 0.761 4±0.047 7
SAGPool	Accuracy/% ROC-AUC	91.30±9.75 0.742 2±0.332 4	63.55±2.55 0.666 1±0.024 6	75.69±2.61 0.807 2±0.030 8	71.89±2.77 0.783 4±0.027 6	71.89±2.77 0.783 4±0.027 6
WaveGIC	Accuracy/% ROC-AUC	99.10±0.41 0.990 5±0.013 0	67.28±2.92 0.721 1±0.021 6	77.18±2.54 0.836 0±0.022 8	76.42±3.10 0.819 0±0.030 1	75.92±5.05 0.797 8±0.047 3
MHAWaveGIC	Accuracy/% ROC-AUC	99.20±0.71 0.991 8±0.010 7	69.68±2.27 0.745 7±0.023 2	78.74±1.89 0.829 9±0.023 4	77.54±2.75 0.823 6±0.026 8	77.99±3.50 0.796 6±0.031 4
ReWaveGIC	Accuracy/% ROC-AUC	95.10±7.96 0.881 4±0.224 5	68.99±2.56 0.734 4±0.026 8	78.66±2.11 0.845 6±0.023 2	76.78±2.06 0.820 8±0.015 6	75.83±4.66 0.792 3±0.043 1
RMAWaveGIC	Accuracy/% ROC-AUC	99.25±0.63 0.989 2±0.014 4	70.16±2.96 0.752 3±0.026 2	78.81±2.90 0.837 3±0.025 3	78.26±2.26 0.830 4±0.023 9	78.26±2.78 0.803 5±0.031 3

[1]	YANG Zhiqiao, ZHANG Ying, WANG Xinjie, ZHANG Dongbo, WANG Yu. Application Research of Improved U-shaped Network in Detection of Retinopathy [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1877-1884.
[2]	XIA Hongbin, XIAO Yifei, LIU Yuan. Long Text Generation Adversarial Network Model with Self-Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1603-1610.
[3]	LI Yunhuan, WEN Jiwei, PENG Li. High Frame Rate Light-Weight Siamese Network Target Tracking [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1405-1416.
[4]	ZHANG Yancao, ZHAO Yuhai, SHI Lan. Multi-feature Based Link Prediction Algorithm Fusing Graph Attention [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1096-1106.
[5]	ZHAO Pengfei, XIE Linbo, PENG Li. Deep Small Object Detection Algorithm Integrating Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 927-937.
[6]	LU Zhongda, ZHANG Chunda, ZHANG Jiaqi, WANG Zifei, XU Junhua. Identification of Apple Leaf Disease Based on Dual Branch Network [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 917-926.
[7]	HUANG Siyuan, ZHAO Yuhai, LIANG Yiming. Code Search Combining Graph Embedding and Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 844-854.
[8]	WANG Yanni, YU Lixian. SSD Object Detection Algorithm with Effective Fusion of Attention and Multi-scale [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(2): 438-447.
[9]	CAI Mingxin, SUN Jing, WANG Bin. Multi-aspect Semantic Trajectory Similarity Computation Model [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(9): 1632-1640.
[10]	CHEN Junfen, ZHANG Ming, ZHAO Jiacheng, XIE Bojun, LI Yan. Deep Clustering Algorithm Based on Denoising and Self-Attention [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(9): 1717-1727.
[11]	ZHANG Han, JIA Tianyuan, LUO Fang, ZHANG Sheng, WU Xia. Study on Predicting Psychological Traits of Online Text by BERT [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(8): 1459-1468.
[12]	CHEN Jiannan, DU Junping, XUE Zhe, KOU Feifei. Accurate Portrait of Big Data of Financial Events Based on Multiple Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(7): 1237-1244.
[13]	FANG Junting, TAN Xiaoyang. Defect Detection of Metal Surface Based on Attention Cascade R-CNN [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(7): 1245-1254.
[14]	WANG Xiaodong, ZHAO Yining, XIAO Haili, WANG Xiaoning, CHI Xuebin. User Behavior Analysis with RNN and Graph Neural Networks [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(5): 838-847.
[15]	WANG Runzheng, GAO Jian, TONG Xin, YANG Mengqi. Research on Malicious Code Family Classification Combining Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(5): 881-892.