Graph Convolutional Network with Adaptive Fusion of Neighborhood Aggregation and Interaction

doi:10.3778/j.issn.1673-9418.2106010

Abstract

Abstract: Graph representation learning technology aims to learn the low dimensional representation vectors for nodes while maintaining the properties of graphs and provide materials for downstream tasks. Most existing algorithms mainly focus on aggregating neighborhood features, lack the ability to capture non-linear information. To solve this issue, this paper proposes a graph convolutional network with adaptive fusion of neighborhood aggregation and interaction (AFAI-GCN). Firstly, a two-channel graph convolutional network is constructed to model the neighborhood aggregation, and the representations generated by modeling are used to calculate the neighborhood interaction items to enhance the learning capability of algorithm. Secondly, the attention mechanism is adopted in the adaptive fusion module to increase the attention on significant information and improve the task correlation of the fused information items. Finally, an information consistency constraint and a difference constraint are applied to enhancing node feature consistency and embedded representation difference. The node classification and visualization tasks are performed on three public citation datasets. Experiments results show that, compared with graph convolutional network (GCN), neighborhood aggregation and interaction graph convolutional network (AIR-GCN) and other algorithms, the classification accuracy of AFAI-GCN is increased by 1.0 to 1.6, 1.1 to 2.4, 0.3 to 0.9 percentage points on Cora, Citeseer and Pubmed datasets, respectively. In the visualization task, the degree of aggregation within clusters is higher and the boundaries of different clusters are clearer. Moreover, the convergence speed of AFAI-GCN is faster and the accuracy curve is smoother in the learning process. All these results indicate that AFAI-GCN is more advanced than the benchmark algorithms.

Key words: graph representation learning, graph convolutional neural network (GCNN), attention mechanism, node classification

摘要： 图表示学习是在保持图中节点性质不变的前提下，获取节点的低维表示向量，为下游任务提供有效的数据支持。现有图表示学习算法大多关注于聚合邻域特征，对于挖掘其他非线性信息关注不足。针对这一问题，提出了自适应融合邻域聚合和邻域交互的图卷积网络AFAI-GCN。首先，采用双通道图卷积网络建模邻域聚合，并利用生成的嵌入表示计算邻域交互项来补充算法学习的信息；然后，结合注意力机制构建自适应融合模块，增加对重要信息的关注，提高融合信息项的任务相关性；最后，通过信息一致性约束和差异性约束增强节点特征一致性和嵌入表示差异性。在三个公共引文数据集上进行了节点分类和可视化任务，结果显示，AFAI-GCN与图卷积网络（GCN）、邻域聚合和交互图卷积网络（AIR-GCN）等算法相比，在Cora、Citeseer、Pubmed数据集上分类准确率分别提高了1.0~1.6个百分点、1.1~2.4个百分点和0.3~0.9个百分点；在可视化任务中团簇内聚合程度更高，不同的团簇边界更清晰；算法学习过程中收敛速度更快，准确率曲线更平滑。实验结果表明该框架较好地提升了基准算法的性能。

关键词: 图表示学习, 图卷积神经网络（GCNN）, 注意力机制, 节点分类

FU Kun, ZHUO Jiaming, GUO Yunpeng, LI Jianing, LIU Qi. Graph Convolutional Network with Adaptive Fusion of Neighborhood Aggregation and Interaction[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(2): 453-466.

富坤, 禚佳明, 郭云朋, 李佳宁, 刘琪. 自适应融合邻域聚合和邻域交互的图卷积网络[J]. 计算机科学与探索, 2023, 17(2): 453-466.

References

[1] 张雁操, 赵宇海, 史岚. 融合图注意力的多特征链接预测算法[J]. 计算机科学与探索, 2022, 16(5): 1096-1106.
ZHANG Y C, ZHAO Y H, SHI L. Multi-feature based link prediction algorithm fusing graph attention[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1096-1106.
[2] YOU J X, YING R, LESKOVEC J. Position-aware graph neural networks[C]//Proceedings of the 36th International Conference on Machine Learning, Long Beach, Jun 9-15, 2019: 7134-7143.
[3] ABU-EL-HAIJA S, PEROZZI B, KAPOOR A, et al. MixHop: higher-order graph convolutional architectures via sparsified neighborhood mixing[C]//Proceedings of the 36th Inter-national Conference on Machine Learning, Long Beach, Jun 9-15, 2019: 21-29.
[4] WU J, HE J R, XU J J. DEMO-Net: degree-specific graph neural networks for node and graph classification[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,Anchorage, Aug 4-8, 2019. New York: ACM, 2019: 406-415.
[5] FAN W Q, MA Y, LI Q, et al. Graph neural networks for social recommendation[C]//Proceedings of the 2019 World Wide Web Conference, San Francisco, May 13-17, 2019. New York: ACM, 2019: 417-426.
[6] 陈洁, 刘洋, 赵姝, 等. 利用多粒度属性网络表示学习进行引文推荐[J]. 计算机科学与探索, 2021, 15(6): 1103-1113.
CHEN J, LIU Y, ZHAO S, et al. Citation recommendation via hierarchical attributed network representation learning[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(6): 1103-1113.
[7] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, Aug 24-27, 2014. New York: ACM, 2014: 701-710.
[8] YANG Z, COHEN W, SALAKHUDINOV R. Revisiting semi-supervised learning with graph embeddings[C]//Proceedings of the 33rd International Conference on Machine Learning,New York, Jun 19-24, 2016: 40-48.
[9] BRUNA J, ZAREMBA W, SZLAM A, et al. Spectral networks and locally connected networks on graphs[C]//Proceedings of the 2nd International Conference on Learning Representa- tions, Banff, Apr 14-16, 2014: 1-14.
[10] DEFFERRARD M, BRESSON X, VANDERGHEYNST P. Convolutional neural networks on graphs with fast localized spectral filtering[C]//Advances in Neural Information Pro-cessing Systems 29, Dec 5-10, 2016. Red Hook: Curran Associates, 2016: 3837-3845.
[11] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[C]//Proceedings of the 5th International Conference on Learning Representations, Toulon, Apr 24-26, 2017: 1-14.
[12] WU F L, SOUZA JR A H, ZHANG T Y, et al. Simplifying graph convolutional networks[C]//Proceedings of the 36th International Conference on Machine Learning, Long Beach, Jun 9-15, 2019: 6861-6871.
[13] VELI?KOVI? P, CUCURULL G, CASANOVA A, et al. Graph attention networks[C]//Proceedings of the 6th Internatio-nal Conference on Learning Representations, Vancouver, Apr 30-May 3, 2018: 1-12.
[14] HAMILTON W L, YING Z T, LESKOVEC J. Inductive representation learning on large graphs[C]//Advances in Neural Information Processing Systems 30, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 1024-1034.
[15] KLICPERA J, BOJCHEVSKI A, GüNNEMANN S, et al. Predict then propagate: graph neural networks meet personalized PageRank[C]//Proceedings of the 7th International Conference on Learning Representations, New Orleans, May 6-9, 2019: 1-15.
[16] XU K, LI C T, TIAN Y L, et al. Representation learning on graphs with jumping knowledge networks[C]//Proceedings of the 35th International Conference on Machine Learning, Stockholmsm?ssan, Jul 10-15, 2018: 5449-5458.
[17] CHEN M, WEI Z W, HUANG Z F, et al. Simple and deep graph convolutional networks[C]//Proceedings of the 37th International Conference on Machine Learning, Jul 13-18, 2020: 1725-1735.
[18] WANG J, DENG Z. A deep graph wavelet convolutional neural network for semi-supervised node classification[J]. arXiv:2102.09780, 2021.
[19] HU F Y, ZHU Y Q, WU S, et al. Graphair: graph representation learning with neighborhood aggregation and interaction[J]. Pattern Recognition, 2021, 112: 107745.
[20] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[21] 潘承瑞, 何灵敏, 胥智杰, 等. 融合知识图谱的双线性图注意力网络推荐算法[J]. 计算机工程与应用, 2021, 57(1): 29-37.
PAN C R, HE L M, XU Z J, et al. Fusion knowledge graph and bilinear graph attention network recommendation algorithm[J]. Computer Engineering and Applications, 2021, 57(1): 29-37.
[22] XIE Y Q, LI S, YANG C, et al. When do GNNs work: understanding and improving neighborhood aggregation[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence, 2020: 1303-1309.
[23] LI Q M, HAN Z C, WU X M. Deeper insights into graph convolutional networks for semi-supervised learning[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 3538-3545.
[24] WESTON J, RATLE F, MOBAHI H, et al. Deep learning via semi-supervised embedding[M]//MONTAVON G, ORR G B, MüLLER K R. LNCS 7700: Neural Networks: Tricks of the Trade. Berlin, Heidelberg: Springer, 2012: 639-655.
[25] GRANDVALET Y, BENGIO Y. Semi-supervised learning by entropy minimization[C]//Advances in Neural Information Processing Systems 17, Vancouver, Dec 13-18, 2004. Red Hook: Curran Associates, 2004: 529-536.
[26] BERTHELOT D, CARLINI N, GOODFELLOW I J, et al. MixMatch: a holistic approach to semi-supervised learning[C]//Advances in Neural Information Processing Systems 32, Vancouver, Dec 8-14, 2019: 5050-5060.
[27] 张晨光, 张燕, 张夏欢. 从希尔伯特-施密特独立性中学习的多标签半监督学习方法[J]. 中国科技论文, 2013, 8(10):998-1002.
ZHANG C G, ZHANG Y, ZHANG X H. Multi-label semi-supervised learning method learnt from Hilbert-Schmidt independence criterion[J]. China Science Paper, 2013, 8(10): 998-1002.
[28] 耿家兴. 基于希尔伯特施密特独立性验证的因果推断方法研究[D]. 衡阳: 南华大学, 2020.
GENG J X. Research on causal inference method based on Schmidt orthogonal matrix verification[D]. Hengyang: Univer-sity of South China, 2020.
[29] WANG X, ZHU M Q, BO D Y, et al. AM-GCN: adaptive multi-channel graph convolutional networks[C]//Proceedings of the 26th ACM SIGKDD Conference on Knowledge Dis-covery and Data Mining. New York: ACM, 2020: 1243-1253.
[30] 孙学全, 冯英浚. 多层感知器的灵敏度分析[J]. 计算机学报, 2001, 24(9): 951-958.
SUN X Q, FENG Y J. Sensitivity analysis of multilayer perceptron[J]. Chinese Journal of Computers, 2001, 24(9): 951-958.
[31] 徐冰冰, 岑科廷, 黄俊杰, 等. 图卷积神经网络综述[J]. 计算机学报, 2020, 43(5): 755-780.
XU B B, CEN K T, HUANG J J, et al. A survey on graph convolutional neural network[J]. Chinese Journal of Computers, 2020, 43(5): 755-780.
[32]VAN DER MAATEN L. Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9: 2579-2605.