融合记忆网络的细粒度实体分类方法

doi:10.3778/j.issn.1673-9418.2103027

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (11): 2565-2574.DOI: 10.3778/j.issn.1673-9418.2103027

融合记忆网络的细粒度实体分类方法

周祺, 陶皖⁺(), 孔超, 崔佰婷

安徽工程大学计算机与信息学院，安徽芜湖 241000

收稿日期:2021-03-26 修回日期:2021-05-14 出版日期:2022-11-01 发布日期:2021-05-18
通讯作者: + E-mail: taowan@ahpu.edu.cn
作者简介:周祺（1997—），女，安徽宿州人，硕士研究生，主要研究方向为云计算、大数据处理。
陶皖（1972—），女，安徽芜湖人，硕士，教授，主要研究方向为大数据、数据分析。
孔超（1986—），男，山东人，博士，副教授，主要研究方向为网络数据管理、流媒体数据处理、社交网络分析、数据挖掘。
崔佰婷（1999—），女，安徽宿州人，主要研究方向为图嵌入、图挖掘。
基金资助:
国家自然科学基金青年基金项目(61902001);安徽省教育厅高校自然科学重点项目(KJ2019A0158);安徽省教育厅高校自然科学重点项目(KJ2019ZD15);国家级大学生创新创业项目(202010363098);国家级大学生创新创业项目(201910363076)

Fine-Grained Entity Classification Method Fused with Memory Network

ZHOU Qi, TAO Wan⁺(), KONG Chao, CUI Baiting

School of Computer and Information, Anhui Polytechnic University, Wuhu, Anhui 241000, China

Received:2021-03-26 Revised:2021-05-14 Online:2022-11-01 Published:2021-05-18
About author:ZHOU Qi, born in 1997, M.S. candidate. Her research interests include cloud computing and big data processing.
TAO Wan, born in 1972, M.S., professor. Her research interests include big data and data analysis.
KONG Chao, born in 1986, Ph.D., associate professor. His research interests include web data management, streaming data processing, social network analysis and data mining.
CUI Baiting, born in 1999. Her research interests include graph embedding and graph mining.
Supported by:
National Natural Science Foundation for Youth of China(61902001);Key Natural Science Project of Education Department of Anhui Province(KJ2019A0158);Key Natural Science Project of Education Department of Anhui Province(KJ2019ZD15);National Innovation and Entrepreneurship Program for College Students(202010363098);National Innovation and Entrepreneurship Program for College Students(201910363076)

摘要/Abstract

摘要：

细粒度实体分类是在给定实体指称后要求为其分配细粒度类型标签的任务。大多数细粒度实体分类采用远程监督的方法，为实体指称分配知识库中实体所对应的全部类型标签，这会引入无关或具体的噪声标签。在远程监督中，将分配与指称上下文无关的类型标签归为无关噪声标签，分配细粒度标签导致在上下文中实体含义不准确的类型标签归为具体噪声标签。为减轻噪声影响，以往采用人工标注、启发式规则剪枝等方法，但存在效率低、缩减训练集规模导致分类模型整体性能变差等问题。通过引入记忆网络，分类模型能深入学习实体指称上下文与类型标签之间的关联性，增强对相似的指称上下文所对应类型标签的记忆表示，有效减轻无关噪声标签的影响。与此同时，利用变形的层次损失函数有效学习类型标签之间的层次关系，从而缓解具体噪声标签的负面影响。此外，结合L2正则化函数防止训练模型对噪声标签的过拟合。在公开数据集上的实验结果表明，提出的方法能够有效缓解无关噪声标签和具体噪声标签对分类模型的消极影响，在准确率、Macro F1值、Micro F1值上表现均优于以往处理标签噪声的方法。

关键词: 细粒度实体分类, 噪声处理, 记忆网络, 类型标签

Abstract:

Fine-grained entity classification is a task that requires a fine-grained type label to be assigned to a given entity mention. Most of the existing fine-grained entity classification uses distant supervision method. All of the type labels corresponding to the entities in the knowledge base are assigned to the entity mention, which will introduce irrelevant or specific noise labels. In distant supervision, type labels that are not related to the entity mention context are classified as out-of-context noise labels, and type labels whose assignment of fine-grained labels leads to inacc-urate entity meaning in the context are classified as overly-specific noise labels. In order to reduce the impact of noise, manual labeling and heuristic pruning methods have been used in the past, but there are some problems such as low efficiency and reducing the size of the training set, which leads to the deterioration of the overall performance of the classification model. By introducing the memory network, the classification model can deeply learn the correlation between the entity mention context and the type label, enhance the memory representation of the type label corresponding to the similar entity mention context, and effectively reduce the influence of out-of-context noise labels. At the same time, transformative hierarchical loss function is used to effectively learn the hierarchical relationship between type labels, so as to alleviate the negative impact of overly-specific noise labels. In addition, using the L2 regularization function can prevent the model from overfitting noise labels. Experimental results on public datasets show that the proposed method can effectively alleviate the negative effects of out-of-context noise labels and overly-specific noise labels on the classification model, and its performance in accuracy, Macro F1 value and Micro F1 value is superior to previous methods for processing noise labels.

Key words: fine-grained entity classification, noise processing, memory network, type label

中图分类号:

TP391.1

周祺, 陶皖, 孔超, 崔佰婷. 融合记忆网络的细粒度实体分类方法[J]. 计算机科学与探索, 2022, 16(11): 2565-2574.

ZHOU Qi, TAO Wan, KONG Chao, CUI Baiting. Fine-Grained Entity Classification Method Fused with Memory Network[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(11): 2565-2574.

图/表 6

图1 两种噪声标签形式

Fig.1 Two forms of noise labels

图2 融合记忆网络的细粒度实体分类模型

Fig.2 Fine-grained entity classification model fused with memory network

图3 记忆网络的具体框架结构

Fig.3 Concrete frame structure of memory network

表1 数据集中的统计数据

Table 1 Statistics in datasets

统计指标	FIGER	BBN
类型数	113	47
最深层次	2	2
mentions-训练	2 009 898	86 078
mentions-测试	563	12 845
干净mentions-训练/%	64.46	75.92
干净mentions-测试/%	88.28	100.00

表2 超参数设置

Table 2 Hyperparameter setting

超参数	FIGER	BBN
$B$	256	256
$l r$	0.002	0.007
$D h$	250	250
$D m$	500	500
$D t$	500	500
$α$	1	1
$β$	0.4	0.4
$λ$	0.000 1	0.000 2

表2 超参数设置

Table 2 Hyperparameter setting

超参数	FIGER	BBN
$B$	256	256
$l r$	0.002	0.007
$D h$	250	250
$D m$	500	500
$D t$	500	500
$α$	1	1
$β$	0.4	0.4
$λ$	0.000 1	0.000 2

表3 Performance comparison of fine-grained entity classification methods 单位：%

Table 3

参考文献 32

[1]	冯建周, 马祥聪. 基于迁移学习的细粒度实体分类方法的研究[J]. 自动化学报, 2020, 46(8): 1759-1766.
	FENG J Z, MA X C. Research on fine-grained entity classi-fication method based on transfer learning[J]. Acta Auto-matica Sinica, 2020, 46(8): 1759-1766.
[2]	LEE C, HWANG Y G, OH H J, et al. Fine-grained named entity recognition using conditional random fields for ques-tion answering[C]// LNCS 4182: Proceedings of the 3rd Asia Information Retrieval Symposium Information Retri-eval Technology, Singapore, Oct 16-18, 2006. Berlin, Heid-elberg: Springer, 2006: 581-587.
[3]	SEKINE S. Extended named entity ontology with attribute information[C]// Proceedings of the 2008 International Con-ference on Language Resources and Evaluation, Marrak-ech, May 26-Jun 1, 2008: 1-6.
[4]	LING X, WELD D S. Fine-grained entity recognition[C]// Proceedings of the 26th AAAI Conference on Artificial Intelligence, Toronto, Jul 22-26, 2012. Menlo Park: AAAI, 2012: 94-100.
[5]	MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]// Proceedings of the 47th Annual Meeting of the Association for Comput-ational Linguistics and the 4th International Joint Confer-ence on Natural Language Processing of the AFNLP, Sing-apore, Aug 2-7, 2009. Stroudsburg: ACL, 2009: 1003-1011.
[6]	YOSEF M A, BAUER S, HOFFART J, et al. HYENA: hie-rarchical type classification for entity names[C]// Proceed-ings of the 24th International Conference on Computat-ional Linguistics, Mumbai, Dec 8-15, 2012. India: Indian Institute of Technology Bombay, 2012: 1361-1370.
[7]	YOGATAMA D, GILLICK D, LAZIC N. Embedding meth-ods for fine grained entity type classification[C]// Proceed-ings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, Jul 26-31, 2015. Stroudsburg: ACL, 2015: 291-296.
[8]	DONG X, GABRILOVICH E, HEITZ G, et al. Knowledge vault: a web-scale approach to probabilistic knowledge fusi-on[C]// Proceedings of the 20th ACM SIGKDD Internat-ional Conference on Knowledge Discovery and Data Mining, New York, Aug 24-27, 2014. New York: ACM, 2014: 601-610.
[9]	DEL CORRO L, ABUJABAL A, GEMULLA R, et al. FINET: context-aware fine-grained named entity typing[C]// Proce-edings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Sep 17-21, 2015. Str-oudsburg: ACL, 2015: 868-878.
[10]	SHIMAOKA S, STENETORP P, INUI K, et al. An atten-tive neural architecture for fine-grained entity type classi-fication[C]// Proceedings of the 5th Workshop on Auto-mated Knowledge Base Construction, San Diego, Jun 17, 2016. Stroudsburg: ACL, 2016: 69-74.
[11]	马建红, 张炳斐, 张少光, 等. 基于主动MCNN-SCRF的新能源汽车命名实体识别[J]. 计算机工程与应用, 2019, 55(7): 23-29. DOI
	MA J H, ZHANG B F, ZHANG S G, et al. New energy vehicle named entity recognition based on active MCNN-SCRF[J]. Computer Engineering and Applications, 2019, 55(7): 23-29.
[12]	盛剑, 向政鹏, 秦兵, 等. 多场景文本的细粒度命名实体识别[J]. 中文信息学报, 2019, 33(6): 80-87.
	SHENG J, XIANG Z P, QIN B, et al. Fine-grained named entity recognition for multi-scene text[J]. Journal of Chinese Information Processing, 2019, 33(6): 80-87.
[13]	王红, 林海舟, 卢林燕. 基于Att_GCN模型的知识图谱推理算法[J]. 计算机工程与应用, 2020, 56(9): 183-189. DOI
	WANG H, LIN H Z, LU L Y. Knowledge graph inference algorithm based on Att_GCN model[J]. Computer Engine-ering and Applications, 2020, 56(9): 183-189.
[14]	胡新棒, 于溆乔, 李邵梅, 等. 基于知识增强的中文命名实体识别[J]. 计算机工程, 2021, 47(11): 84-92.
	HU X B, YU G Q, LI S M, et al. Chinese named entity recognition based on knowledge enhancement[J]. Computer Engineering, 2021, 47(11): 84-92.
[15]	西尔艾力·色提, 艾山·吾买尔, 王路路, 等. 结合单词-字符引导注意力网络的中文旅游文本命名实体识别[J]. 计算机工程, 2021, 47(2): 39-45.
	XIERAILI S, AISHAN W, WANG L L, et al. Named entity recognition in Chinese tourism text based on wordcharacter guided attention network[J]. Computer Engineering, 2021, 47(2): 39-45.
[16]	LAWRENCE N D, SCHÖLKOPF B. Estimating a kernel Fisher discriminant in the presence of label noise[C]// Proceedings of the 18th International Conference on Mach-ine Learning, Williamstown, Jun 28-Jul 1, 2001. San Franci-sco: Morgan Kaufmann, 2001: 306-313.
[17]	GILLICK D, LAZIC N, GANCHEV K, et al. Context-dependent fine-grained entity type tagging[J]. arXiv:1412.1820, 2014.
[18]	REN X, HE W, QU M, et al. AFET: automatic fine-grained entity typing by hierarchical partial-label embedding[C]// Pro-ceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Strou-dsburg: ACL, 2016: 1369-1378.
[19]	ABHISHEK A, ANAND A, AWEKAR A. Fine-grained entity type classification by jointly learning representations and label embeddings[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Comput-ational Linguistics, Valencia, Apr 3-7, 2017. Stroudsburg: ACL, 2017: 797-807.
[20]	XU P, BARBOSA D. Neural fine-grained entity type classi-fication with hierarchy-aware loss[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Langu-age Technologies, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 16-25.
[21]	CHEN B, GU X, HU Y, et al. Improving distantly-supervised entity typing with compact latent space clustering[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Minneapolis, Jun 2-7, 2019. Stroudsburg: ACL, 2019: 2862-2872.
[22]	XIN J, ZHU H, HAN X, et al. Put it back: entity typing with language model enhancement[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Langu-age Processing, Brussels, Oct 31-Nov 4, 2018. Strouds-burg: ACL, 2018: 993-998.
[23]	ZHANG H, LONG D, XU G, et al. Learning with noise: improving distantly-supervised fine-grained entity typing via automatic relabeling[C]// Proceedings of the 29th Internati-onal Joint Conference on Artificial Intelligence, Yokohama, Jul 2020: 3808-3815.
[24]	XIA S, WANG G, CHEN Z, et al. Complete random forest based class noise filtering learning for improving the gene-ralizability of classifiers[J]. IEEE Computer Architecture Letters, 2019, 31(11): 2063-2078.
[25]	ZHANG W, WANG D, TAN X. Robust class-specific auto-encoder for data cleaning and classification in the presence of label noise[J]. Neural Processing Letters, 2019, 50(2): 1845-1860. DOI URL
[26]	WEI Y, GONG C, CHEN S, et al. Harnessing side inform-ation for classification under label noise[J]. IEEE Transa-ctions on Neural networks and Learning Systems, 2019, 31(9): 3178-3192.
[27]	ZHOU B, KHASHABI D, TSAI C T, et al. Zero-shot open entity typing as type-compatible grounding[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Strouds-burg: ACL, 2018: 2065-2076.
[28]	DAI H, DU D, LI X, et al. Improving fine-grained entity typing with entity linking[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 6209-6214.
[29]	PAN X, CASSIDY T, HERMJAKOB U, et al. Unsuper-vised entity linking with abstract meaning representation[C]// Proceedings of the 2015 Conference of the North Ame-rican Chapter of the Association for Computational Lingu-istics:Human Language Technologies, Denver, May 31-Jun 5, 2015. Stroudsburg: ACL, 2015: 1130-1139.
[30]	ZHOU P, SHI W, TIAN J, et al. Attention-based bidire-ctional long short-term memory networks for relation classi-fication[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Aug 7-12, 2016. Stroudsburg: ACL, 2016: 207-212.
[31]	WEISCHEDEL R, BRUNSTEIN A. BBN pronoun corefer-ence and entity type corpus[EB/OL]. [2021-01-06]. https://doi.org/10.35111/9fx9-gz10.
[32]	PENNINGTON J, SOCHER R, MANNING C D. GloVe: global vectors for word representation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Lan-guage Processing, Doha, Oct 25-29, 2014. Stroudsburg: ACL, 2014: 1532-1543.

融合记忆网络的细粒度实体分类方法

Fine-Grained Entity Classification Method Fused with Memory Network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 32

相关文章 9

编辑推荐

Metrics

[1]	唐晨, 赵杰煜, 叶绪伦, 郑阳, 俞书世. 动态图的链接预测模型[J]. 计算机科学与探索, 2022, 16(10): 2365-2376.
[2]	李兴秀, 唐建军, 华晶. 结合CNN与双向LSTM的心律失常分类[J]. 计算机科学与探索, 2021, 15(12): 2353-2361.
[3]	程琪芩，万良. BiLSTM在跨站脚本检测中的应用研究[J]. 计算机科学与探索, 2020, 14(8): 1338-1347.
[4]	韩鑫鑫，贲可荣，张献. 军用软件测试领域的命名实体识别技术研究[J]. 计算机科学与探索, 2020, 14(5): 740-748.
[5]	张周彬，相艳，梁俊葛，杨嘉林，马磊. 利用位置增强注意力机制的属性级情感分类[J]. 计算机科学与探索, 2020, 14(4): 619-627.
[6]	刘辰，肖志勇，杜年茂. 改进的卷积神经网络在医学图像分割上的应用[J]. 计算机科学与探索, 2019, 13(9): 1593-1603.
[7]	李冬梅，檀稳. 植物属性文本的命名实体识别方法研究[J]. 计算机科学与探索, 2019, 13(12): 2085-2093.
[8]	高雅，江国华，秦小麟，王钟毓. 基于LSTM的移动对象位置预测算法[J]. 计算机科学与探索, 2019, 13(1): 23-34.
[9]	王凯，洪宇，邱盈盈，姚建民，周国栋. 融合上下文依赖和句子语义的事件线索检测研究[J]. 计算机科学与探索, 2018, 12(3): 423-431.