Named Entity Recognition of Wheat Diseases and Pests Fusing ALBERT and Rules

doi:10.3778/j.issn.1673-9418.2203129

Abstract

Abstract: Named entity recognition of wheat diseases and pests is a key step to building a knowledge graph. Aiming at the problems of lack of training data, complex entity structure, diverse entity types and uneven entity distribution in wheat diseases and pests field, under the promise of fully mining the implicit knowledge, two data augmentation methods are used to expand sentence semantic information, and to construct the corpus WpdCNER (wheat pests and diseases Chinese named entity recognition) and the field lexicon WpdDict (wheat pests and diseases dictionary). And 16 categories of entities are defined with the field experts’ guidance. Meanwhile, Chinese named entity recognition model based on rules amendment WPD-RA (wheat pests and disease-rules amendment model) is proposed. This model is carried out entity recognition based on ALBERT+BiLSTM+CRF (a lite bi-directional encoder representation from transformer + bi-directional long short-term memory + conditional random field), and specific rules are defined to amend entity boundaries after recognition. The WPD-RA model achieves the best results with 94.72% precision, 95.23% recall, and 94.97% F1. Its precision is increased by 1.71 percentage points, recall is increased by 0.34 percentage points, and F1 is increased by 1.03 percentage points, compared with the model without rules. Experimental results show that the model can effectively recognize named entities in wheat diseases and pests field, and its performance is better than other models. The proposed model provides a reference idea for named entity recognition task in other fields such as food safety and biology.

Key words: wheat diseases and pests, data augmentation, named entity recognition (NER), ALBERT, rules amendment

摘要： 小麦病虫害中文命名实体识别是构建该领域知识图谱的关键步骤，针对小麦病虫害领域训练数据匮乏、实体结构复杂、实体类型多样及实体分布不均匀等问题，在充分挖掘隐含知识的前提下，采用了两种数据增广方法扩充句子语义信息，构建了小麦病虫害实体识别语料库WpdCNER及其领域词典WpdDict，并在领域专家的指导下定义了16类实体；同时提出了一种基于规则修正的中文命名实体识别模型WPD-RA，该模型基于轻量级BERT＋双向长短期记忆网络＋条件随机场（ALBERT＋BiLSTM＋CRF）进行实体识别，并在识别后定义具体规则校准实体边界。融合规则后的ALBERT+BiLSTM+CRF模型取得了最好的识别结果，准确率为94.72%，召回率为95.23%，[F1]值为94.97%，相比不加规则的识别结果，其准确率、召回率、[F1]值分别增加了1.71个百分点、0.34个百分点、1.03个百分点。实验结果表明，该方法能有效识别小麦病虫害领域命名实体，识别性能优于其他模型，为食品安全、生物等其他领域命名实体识别提供了一种可借鉴的思路。

关键词: 小麦病虫害, 数据增广, 命名实体识别（NER）, ALBERT, 规则修正

LIU Hebing, ZHANG Demeng, XIONG Shufeng, MA Xinming, XI Lei. Named Entity Recognition of Wheat Diseases and Pests Fusing ALBERT and Rules[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(6): 1395-1404.

刘合兵, 张德梦, 熊蜀峰, 马新明, 席磊. 融合ALBERT与规则的小麦病虫害命名实体识别[J]. 计算机科学与探索, 2023, 17(6): 1395-1404.

References

[1] 任媛, 于红, 杨鹤, 等. 融合注意力机制与BERT+BiLSTM+ CRF模型的渔业标准定量指标识别[J]. 农业工程学报, 2021, 37(10): 135-141.
REN Y, YU H, YANG H, et al. Recognition of quantitative indicator of fishery standard using attention mechanism and the BERT+BiLSTM+CRF model[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020, 37(10): 135-141.
[2] 赵山, 罗睿, 蔡志平. 中文命名实体识别综述[J]. 计算机科学与探索, 2022, 16(2): 296-304.
ZHAO S, LUO R, CAI Z P. Survery of Chinese named entity recognition[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(2): 296-304.
[3] 刘新亮, 张梦琪, 谷情, 等. 基于BERT-CRF模型的生鲜蛋供应链命名实体识别[J]. 农业机械学报, 2021, 52(S1): 519-525.
LIU X L, ZHANG M Q, GU Q, et al. Named entity recog-nition of fresh egg supply chain based on BERT-CRF arch-itecture[J]. Transactions of the Chinese Society of Agricul-tural Machinery, 2021, 52(S1): 519-525.
[4] 羊艳玲, 李燕, 钟昕妤, 等. 基于BiLSTM-CRF的中医医案命名实体识别[J]. 中医药信息, 2021, 38(11): 15-21.
YANG Y L, LI Y, ZHONG X Y, et al. Named entity recognition of TCM medical records based on BiLSTM-CRF[J]. Information on Traditional Chinese Medicine, 2021, 38(11): 15-21.
[5] 许力, 李建华. 基于BERT和BiLSTM-CRF的生物医学命名实体识别[J]. 计算机工程与科学, 2021, 43(10): 1873-1879.
XU L, LI J H. Biomedical named entity recognition based on BERT and BiLSTM-CRF[J]. Computer Engineering and Science, 2021, 43(10): 1873-1879.
[6] 沈同平, 俞磊, 金力, 等. 基于BERT-BiLSTM-CRF模型的中文实体识别研究[J]. 齐齐哈尔大学学报(自然科学版), 2022, 38(1): 26-32.
SHEN T P, YU L, JIN L, et al. Chinese entity recognition based on BERT-BiLSTM-CRF model[J]. Journal of Qiqihar University (Natural Science Edition), 2022, 38(1): 26-32.
[7] MALARKODI C S, LEX E, DEVI S L. Named entity recog-nition for the agricultural domain[J]. Research in Computing Science, 2016, 117(1): 121-132.
[8] GUO X, ZHOU H, SU J, et al. Chinese agricultural disea-ses and pests named entity recognition with multiscale local context features and self-attention mechanism[J]. Computers and Electronics in Agriculture, 2020, 179(5): 105830.
[9] 闫丽华. 基于知识图谱的葡萄病虫害自动问答系统[D]. 咸阳: 西北农林科技大学, 2021.
YAN L H. Automatic question answering system for grape diseases and pests based on knowledge graph[D]. Xian-yang: Northwest Agriculture and Forestry University of Science and Technology, 2021.
[10] 于合龙, 沈金梦, 毕春光, 等. 基于知识图谱的水稻病虫害智能诊断系统[J]. 华南农业大学学报, 2021, 42(5): 105-116.
YU H L, SHEN J M, BI C G, et al. Intelligent diagnosis system for rice diseases and pests based on knowledge graph[J]. Journal of South China Agricultural University, 2021, 42(5): 105-116.
[11] 李悦. 农作物病虫害知识图谱构建研究[D]. 北京: 中国农业科学院, 2021.
LI Y. Research on the construction of knowledge graph of crop diseases and pests[D]. Beijing: Chinese Academy of Agricultural Sciences, 2021.
[12] 任妮, 鲍彤, 沈耕宇, 等. 基于深度学习的细粒度命名实体识别研究——以番茄病虫害为例[J]. 情报科学, 2021, 39(11): 96-102.
REN N, BAO T, SHEN G Y, et al. Fine-grained named ent-ity recognition based on deep learning: a case study of tomato diseases and pests[J]. Information Science, 2021, 39(11): 96-102.
[13] 郑泳智, 吴惠粦, 朱定局, 等. 基于荔枝和龙眼病虫害知识图谱的问答系统[J]. 计算机与数字工程, 2021, 49(12): 2618-2622.
ZHENG Y Z, WU H L, ZHU D J, et al. Question and answer system based on the knowledge graphs of litchi and longan diseases and insect pests[J]. Computer & Digital Engineering, 2021, 49(12): 2618-2622.
[14] 杨锦锋, 关毅, 何彬, 等. 中文电子病历命名实体和实体关系语料库构建[J]. 软件学报, 2016, 27(11): 2725-2746.
YANG J F, GUAN Y, HE B, et al. Corpus construction for named entity and entity relations on Chinese electronic medical records[J]. Journal of Software, 2016, 27(11): 2725- 2746.
[15] 沈利言, 姜海燕, 胡滨, 等. 水稻病虫草害与药剂实体关系联合抽取算法[J]. 南京农业大学学报, 2020, 43(6): 1151-1161.
SHEN L Y, JIANG H Y, HU B, et al. A study on joint entity recognition and relation extraction for rice diseases pests weeds and drugs[J]. Journal of Nanjing Agricultural Univer-sity, 2020, 43(6): 1151-1161.
[16] 石教祥, 朱礼军, 望俊成, 等. 面向少量标注数据的命名实体识别研究[J]. 情报工程, 2020, 6(4): 37-50.
SHI J X, ZHU L J, WANG J C, et al. Research on named entity recognition from sparsely labeled data[J]. Technol-ogy Intelligence Engineering, 2020, 6(4): 37-50.
[17] 杨鹤, 于红, 刘巨升, 等. 基于BERT+BiLSTM+CRF深度学习模型和多元组合数据增广的渔业标准命名实体识别[J]. 大连海洋大学学报, 2021, 36(4): 661-669.
YANG H, YU H, LIU J S, et al. Fishery standard named entity recognition based on BERT+BiLSTM+CRF deep learning model and multivariate combination data augmen-tation[J]. Journal of Dalian Ocean University, 2021, 36(4): 661-669.
[18] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient esti-mation of word representations in vector space[J]. arXiv:1301.3781, 2013.
[19] PENNINGTON J, SOCHER R, MANNING C D. GloVe: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Lan-guage Processing, Doha, Oct 25-29, 2014. Stroudsburg: ACL, 2014: 1532-1543.
[20] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2017, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 5998-6008.
[21] DEVLIN J, CHANG M W, LEE K, et al. BERT: pretrai-ning of deep bidirectional transformers for language under-standing[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Com-putational Linguistics: Human Language Technologies, Min-neapolis, Jun 2-7, 2019. Stroudsburg: ACL, 2019: 4171-4186.
[22] LAN Z H, CHEN M D, GOODMAN S, et al. ALBERT: a lite BERT for self-supervised learning of language represe-ntations[C]//Proceedings of the 8th International Conference on Learning Representations, Addis Ababa, Apr 26-30, 2020: 1-17.
[23] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[24] GRAVES A, SCHMIDHUBER J. Framewise phoneme class-ification with bidirectional LSTM and other neural network architectures[J]. Neural Networks, 2005, 18(5/6): 602-610.
[25] LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields: probabilistic models for seg-menting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning, Wil-liams College, Jun 28-Jul 1, 2001. San Mateo: Morgan Kau-fmann, 2001: 282-289.