Graph Neural Network Defense Combined with Contrastive Learning

doi:10.3778/j.issn.1673-9418.2204109

Abstract

Abstract: Although graph neural networks have achieved good performance in the field of graph representation learning, recent studies have shown that graph neural networks are more susceptible to adversarial attacks on graph structure, namely, by adding well-designed perturbations to the graph structure, the performance of graph neural network drops sharply. At present, although mainstream graph structure denoising methods can effectively resist graph structure adversarial attacks, due to the uncertainty of the degree of adversarial attack on the input graph, such methods are prone to more misidentifications when the input graph is not attacked or the attack intensity is small, which damages the prediction results of the graph neural network. To alleviate this problem, this paper proposes a graph neural network defense method combined with contrastive learning (CLD-GNN). Firstly, on the basis of feature similarity denoising, according to the characteristics of label inconsistency between edge endpoints after attack, the label propagation algorithm is used to obtain pseudo-labels of unlabeled nodes, and possible perturbed edges are removed according to the pseudo-label inconsistency between endpoints, resulting in the purification graph. Then, graph convolution is performed on the purification and input graph respectively. Finally, contrastive learning is applied to aligning the predicted label information on the two graphs and modifying the feature representation of the purification graph nodes. Defense experiments are conducted on 3 benchmark datasets and 2 attack scenarios for graph adversarial attacks. Experimental results show that CLD-GNN not only solves the problem of graph denoising methods and prediction effects, but also exhibits excellent defense ability.

Key words: graph neural network, graph structure adversarial attacks, graph neural network defense

摘要： 尽管图神经网络在图表示学习领域中已取得了较好性能，然而研究表明图神经网络易受图结构对抗攻击，即通过对图结构添加精心设计的扰动会使图神经网络的性能急剧下降。目前，主流的图结构去噪方法虽能有效防御图结构对抗攻击，但由于输入图遭受对抗攻击程度的不确定性，该类方法在输入图未受攻击或攻击强度较小时易产生较多误识别，反而损害图神经网络预测结果。对此，提出一种结合对比学习的图神经网络防御方法（CLD-GNN）。该方法在基于特征相似性去噪的基础上，根据攻击后连边端点间标签不一致的特点，使用标签传播算法获取未标记节点的伪标签，基于连边端点间伪标签的不一致性去除可能的攻击边，获得净化图；然后分别对净化图和输入图进行图卷积；最后应用对比学习对齐两个图上的预测标签信息，修正净化图节点的特征表示。在图对抗攻击的3个基准数据集、2个攻击场景上进行防御实验，实验结果表明，CLD-GNN不仅缓解了图去噪方法损害节点分类预测效果问题，而且还能在较强攻击场景下表现出较优异的防御能力。

关键词: 图神经网络, 图结构对抗攻击, 图神经网络防御

CHEN Na, HUANG Jincheng, LI Ping. Graph Neural Network Defense Combined with Contrastive Learning[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(8): 1949-1960.

陈娜, 黄金诚, 李平. 结合对比学习的图神经网络防御方法[J]. 计算机科学与探索, 2023, 17(8): 1949-1960.

References

[1] TANG J, QU M, MEI Q Z. PTE: predictive text embedding through large-scale heterogeneous text networks[C]//Pro-ceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Aug 10-13, 2015. New York: ACM, 2015: 1165-1174.
[2] CHEN J, ZHANG J, XU X, et al. E-LSTM-D: a deep learning framework for dynamic network link prediction[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems,2019, 51(6): 3699-3712.
[3] XUAN Q, WANG J, ZHAO M, et al. Subgraph networks with application to structural feature space expansion[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 33(6): 2776-2789.
[4] ZUGNER D, AKBARNEJAD A, GUNNEMANN S. Adver-sarial attacks on neural networks for graph data[C]//Pro-ceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, London, Aug 19-23, 2018. New York: ACM, 2018: 2847-2856.
[5] ZHU D Y, ZHANG Z W, CUI P, et al. Robust graph con-volutional networks against adversarial attacks[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Anchorage, Aug 4-8, 2019. New York: ACM, 2019: 1399-1407.
[6] TANG X F, LI Y D, SUN Y W, et al. Transferring robustness for graph neural network against poisoning attacks[C]//Proceedings of the 13th International Conference on Web Search and Data Mining, Houston, Feb 3-7, 2020. New York: ACM, 2020: 600-608.
[7] GEISLER S, ZUGNER D, GüNNEMANN S. Reliable graph neural networks via robust aggregation[C]//Advances in Neural Information?Processing?Systems?33,?Dec?6-12,?2020:13272-13284.
[8] CHEN L, LI J T, PENG Q B, et al. Understanding structural vulnerability in graph convolutional networks[C]//Proceedings of the 30th International Joint Conference on Artificial Intelligence, Montreal, Aug 19-27, 2021: 2249-2255.
[9] ZUGNER D, GUNNEMANN S. Adversarial attacks on graph neural networks via meta learning[C]//Proceedings of the 7th International Conference on Learning Representations, New Orleans, May 6-9, 2019: 1-15.
[10] WU H J, WANG C, TYSHETSKIY Y, et al. Adversarial examples for graph data: deep insights into attack and defense[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 4816-4823.
[11] ENTEZARI N, SABA A, DARVISHZADEH A, et al. All you need is low (rank): defending against adversarial attacks on graphs[C]//Proceedings of the 13th International Conference on Web Search and Data Mining, Houston, Feb 3-7, 2020. New York: ACM, 2020: 169-177.
[12] JIN W, MA Y, LIU X R, et al. Graph structure learning for robust graph neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, California, Jul 6-10, 2020. New York: ACM, 2020: 66-74.
[13] WU Z R, XIONG Y J, YU S X, et al. Unsupervised feature learning via non-parametric instance discrimination[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake, Jun 18-23, 2018. Piscataway: IEEE, 2018: 3733-3742.
[14] CHEN T, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[C]//Proceedings of the 37th International Conference on Machine Learning, Jul 13-18, 2020: 1597-1607.
[15] HE K M, FAN H Q, WU Y X, et al. Momentum contrast for unsupervised visual representation learning[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 9726-9735.
[16] KIPF T, WELLING M. Variational graph auto-encoders[J]. arXiv:1611.07308, 2016.
[17] YOU Y N, CHEN T L, SUI Y D, et al. Graph contrastive learning with augmentations[C]//Advances in Neural Information Processing Systems 33, Dec 6-12, 2020: 5812-5823.
[18] SUN F Y, HOFFMANN J, VERMA V, et al. InfoGraph: unsupervised and semi-supervised graph-level representation learning via mutual information maximization[C]//Proceedings of the 8th International Conference on Learning Repre-sentations, Addis Ababa, Apr 26-30, 2020: 1-16.
[19] HASSANI K, AHMADI A H. Contrastive multi-view representation learning on graphs[C]//Proceedings of the 37th International Conference on Machine Learning, Jul 13-18, 2020: 4116-4126.
[20] GUO J Y, LI S Y, ZHAO Y, et al. Learning robust repres-entation through graph adversarial contrastive learning[C]//LNCS 13245: Proceedings of the 27th International Con-ference on Database Systems for Advanced Applications, Apr 11-14, 2022. Cham: Springer, 2022: 682-697.
[21] KLICPERA J, BOJCHEVSKI A, GUNNEMANN S. Predict then propagate: graph neural networks meet personalized PageRank[C]//Proceedings of the 7th International Conference on Learning Representations, New Orleans, May 6-9, 2019: 1-15.
[22] KIPF T, WELLING M. Semi-supervised classification with graph convolutional networks[C]//Proceedings of the 5th International Conference on Learning Representations, Toulon, Apr 24-26, 2017: 1-14.
[23] BISHOP C M, NASRABADI N M. Pattern recognition and machine learning[M]. Berlin, Heidelberg: Springer, 2006.
[24] LI Y X, JIN W, XU H, et al. DeepRobust: a pytorch library for adversarial attacks and defenses[J]. arXiv:2005.06149,2020.
[25] WU F, SOUZA A, ZHANG T Y, et al. Simplifying graph convolutional networks[C]//Proceedings of the 36th Inter-national Conference on Machine Learning, Long Beach, Jun 9-15, 2019: 6861-6871.
[26] CHEN M, WEI Z W, HUANG Z F, et al. Simple and deep graph convolutional networks[C]//Proceedings of the 37th International Conference on Machine Learning, Jul 13-18, 2020: 1725-1735.