Hierarchical Labels Guided Attributed Network Embedding

doi:10.3778/j.issn.1673-9418.2006069

Abstract

Abstract:

Network embedding, aiming to learn low dimensional vectors for nodes while preserving important properties of the network, benefits plenty of network applications. Most existing works focus on network structure, node attribute information or label information. However, many real world networks are often associated with abundant hierarchical labels information, which is potentially valuable in seeking more effective network embedding. Since the information between labels in different levels is hard to correlate or inherit, how to make reasonable use of hierarchical label information to learn more efficient network embedding is still an urgent problem. To address the above issues, a novel hierarchical labels guided attributed network embedding framework (HLANE) is proposed. This framework incorporates hierarchical labels information into network embedding with the help of hierarchical attention layer. HLANE first captures structure and/or attributes information by any existing network embedding method to initialize embeddings. Then the hierarchical attention layer, which builds the connection between the parent labels and the child labels, integrates the hierarchical labels information to guide initial embedding so that it generates hierarchical embedding and entire embedding results with hierarchical labels information. Experiments on real-world datasets demonstrate that the proposed method achieves significantly better performance compared with the state-of-the-art embedding algorithms.

Key words: network embedding, attributed network, hierarchical labels, hierarchical attention

摘要：

网络嵌入是在保持网络性质不变的前提下，将节点转换为低维向量，以便下游任务的求解。现有网络嵌入方法的研究大多关注于网络结构、节点属性信息或单层次标签信息等方面。然而，许多真实世界的网络节点通常具有丰富的层次标签信息，这些层次标签信息对获取高效的网络嵌入具有重要价值。由于不同层次的标签之间的信息很难相互关联或继承，如何合理利用层次标签信息进行网络嵌入，获得更高效的向量表示是亟待研究的问题。针对上述问题，提出了一种新的基于层次标签的属性网络嵌入框架（HLANE），该框架利用层次注意力机制将层次标签信息融入网络嵌入中。HLANE框架首先通过现有的网络嵌入方法获取结构和/或属性信息初始化节点的嵌入向量。然后通过层次注意力机制层建立多层次标签的父节点和子节点之间的联系，并依此指导网络节点初始化嵌入向量在不同层次的学习，最终生成网络节点的多层次嵌入向量表示。在真实数据集上的实验表明，与对比算法相比，HLANE框架具有更好的网络节点嵌入表示。

关键词: 网络嵌入, 属性网络, 层次标签, 层次注意力机制

CHEN Jie, CHEN Jialin, ZHAO Shu, ZHANG Yanping. Hierarchical Labels Guided Attributed Network Embedding[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(7): 1279-1288.

陈洁, 陈嘉琳, 赵姝, 张燕平. 层次标签引导的属性网络嵌入[J]. 计算机科学与探索, 2021, 15(7): 1279-1288.

References

[1] XIAO L, CHEN B L, HUANG X, et al. Multi-label text classification method based on label semantic information[J]. Journal of Software, 2020, 31(4): 1079-1089.
肖琳, 陈博理, 黄鑫, 等. 基于标签语义注意力的多标签文本分类[J]. 软件学报, 2020, 31(4): 1079-1089.
[2] ZENG Y F, MU Q L, ZHOU L, et al. Graph embedding based session perception model for next-click recommendation[J]. Journal of Computer Research and Development, 2020, 57(3): 590-603.
曾义夫, 牟其林, 周乐, 等. 基于图表示学习的会话感知推荐模型[J]. 计算机研究与发展, 2020, 57(3): 590-603.
[3] HUANG W, CHEN E H, LIU Q, et al. Hierarchical multi-label text classification: an attention-based recurrent network approach[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, Nov 3-7, 2019. New York: ACM, 2019: 1051-1060.
[4] MA J X, CUI P, WANG X, et al. Hierarchical taxonomy aware network embedding[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, Aug 19-23, 2018. New York: ACM, 2018: 1920-1929.
[5] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations[C]//Proceedings of the 2014 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, Aug 24-27, 2014. New York: ACM, 2014: 701-710.
[6] GROVER A, LESKOVEC J. node2vec: scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, Aug 13-17, 2016. New York: ACM, 2016: 855-864.
[7] TANG J, QU M, WANG M Z, et al. LINE: large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web, Florence, May 18-22, 2015. New York: ACM, 2015: 1067-1077.
[8] QIU J Z, DONG Y X, MA H, et al. Network embedding as matrix factorization: unifying DeepWalk, LINE, PTE, and node2vec[C]//Proceedings of the 11th ACM International Conference on Web Search and Data Mining, Marina Del Rey, Feb 5-9, 2018. New York: ACM, 2018: 459-467.
[9] LI Y, WANG Y, ZHANG T T, et al. Learning network embedding with community structural information[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 2937- 2943.
[10] MIKOLOV T, SUTSKEVERI, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, Dec 5-8, 2013. Red Hook: Curran Associates, 2013: 3111-3119.
[11] YANG C, LIU Z Y, ZHAO D L, et al. Network representation learning with rich text information[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Jul 25-31, 2015. Menlo Park: AAAI, 2015: 2111-2117.
[12] ZHANG Z, YANG H X, BU J J, et al. ANRL: attributed network representation learning via deep neural networks[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Jul 13-19, 2018: 3155-3161.
[13] LIU J, HE Z C, WEI L, et al. Content to node: self-translation network embedding[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, Aug 19-23, 2018. New York: ACM, 2018: 1794-1802.
[14] MENG Z Q, LIANG S S, BAO H Y, et al. Co-embedding attributed networks[C]//Proceedings of the 12th ACM International Conference on Web Search and Data Mining, Melbourne, Feb 11-15, 2019. New York: ACM, 2019: 393-401.
[15] GAO H C, HUANG H. Deep attributed network embedding[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Jul 13-19, 2018: 3364-3370.
[16] DONG Y Q, WANG X, LIU Y B, et al. Building network domain knowledge graph from heterogeneous YANG models[J]. Journal of Computer Research and Development, 2020, 57(4): 699-708.
董永强, 王鑫, 刘永博, 等. 异构YANG模型驱动的网络领域知识图谱构建[J]. 计算机研究与发展, 2020, 57(4): 699-708.
[17] SHI C, HU B B, ZHAO W X, et al. Heterogeneous information network embedding for recommendation[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 31(2): 357-370.
[18] CEN Y K, ZOU X, ZHANG J W, et al. Representation lear-ning for attributed multiplex heterogeneous network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, Aug 4-8, 2019. New York: ACM, 2019: 1358-1368.
[19] PAN S R, WU J, ZHU X Q, et al. Tri-party deep network representation[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, Jul 9-15, 2016. Menlo Park: AAAI, 2016: 1895-1901.
[20] HUANG X, LI J D, HU X. Label informed attributed network embedding[C]//Proceedings of the 10th ACM International Conference on Web Search and Data Mining, Cambridge, Feb 6-10, 2017. New York: ACM, 2017: 731-739.
[21] KIPFAND T N, WELLING M. Semi-supervised classification with graph convolutional networks[C]//Proceedings of the 5th International Conference on Learning Representations, Toulon, Apr 24-26, 2017: 1-14.
[22] HAMILTON W L, YING Z T, LESKOVEC J. Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 1024-1034.
[23] HU F Y, ZHU Y Q, WU S, et al. Hierarchical graph convolutional networks for semi-supervised node classification[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 4532-4539.
[24] XU B B, SHEN H W, CAO Q, et al. Graph wavelet neural network[C]//Proceedings of the 7th International Conference on Learning Representations, New Orleans, May 6-9, 2019: 1-13.
[25] SEN P, NAMATA G, BILGIC M, et al. Collective classification in network data[J]. AI Magazine, 2008, 29(3): 93-106.
[26] GRADY L. Random walks for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(11): 1768-1783.