[1]钟茂生,吴佳华,罗 玮,等.面向低资源命名实体识别的BiLSTM-Att-BCRF模型[J].江西师范大学学报(自然科学版),2022,(05):460-467.[doi:10.16357/j.cnki.issn1000-5862.2022.05.04]
　ZHONG Maosheng,WU Jiahua,LUO Wei,et al.The BiLSTM-Att-BCRF Model for Low Resource Named Entity Recognition[J].Journal of Jiangxi Normal University:Natural Science Edition,2022,(05):460-467.[doi:10.16357/j.cnki.issn1000-5862.2022.05.04]
点击复制

面向低资源命名实体识别的BiLSTM-Att-BCRF模型()
分享到：

《江西师范大学学报》（自然科学版）[ISSN:1006-6977/CN:61-1281/TN]

卷:
期数:
2022年05期
页码:
460-467
栏目:
信息科学与技术
出版日期:
2022-09-25

文章信息/Info

Title:
The BiLSTM-Att-BCRF Model for Low Resource Named Entity Recognition
文章编号:
1000-5862(2022)05-0460-08
作者:
钟茂生; 吴佳华; 罗 玮; 吴水秀
(江西师范大学计算机信息工程学院,江西 南昌 30022)
Author(s):
ZHONG Maosheng; WU Jiahua; LUO Wei; WU Shuixiu
(School of Computer and Information Engineering,Jiangxi Normal University,Nanchang Jiangxi 330022,China)
关键词:
低资源命名实体识别;  神经网络;  伯努利分布;  自注意力机制
Keywords:
low resource named entity recognition;  neural network;  Bernoulli distribution;  self-attention mechanism
分类号:
TP 391
DOI:
10.16357/j.cnki.issn1000-5862.2022.05.04
文献标志码:
A
摘要:
在低资源场景下,由于受训练数据量少的限制,现有模型的参数不能拟合到预期效果,所以导致模型识别实体的性能不佳.该文提出一种融入伯努利分布(Bernoulli distribution)的新型损失函数,使模型能较好拟合数据.此外,该文在BiLSTM-CRF模型基础上融合多层字符特征信息和自注意力机制,并结合基于伯努利分布的新型损失函数,构建了BiLSTM-Att-BCRF模型.BiLSTM-Att-BCRF模型在20%的CONLL2003和20%的BC5CDR的数据集上,F1值在BiLSTM-CRF模型基础上分别提升了7.00%和4.08%,能较好地适应低资源命名实体识别任务.
Abstract:
In low-resource scenarios,the existing models are limited by the small amount of training data,and the parameters are not fitted to the expected effect,resulting in poor performance of the model in recognizing entities.In this paper,a new loss function incorporating Bernoulli distribution is proposed to allow the model to fit the data better.In addition,a BiLSTM-Att-BCRF model based on the BiLSTM-CRF model is constructed by fusing multi-layer character feature information and self-attention mechanism,combined with a novel loss function based on Bernoulli distribution.The BiLSTM-Att-BCRF model proposed in this paper improves the F1 values by 7.00% and 4.08% based on the BiLSTM-CRF model on the datasets of 20% CONLL2003 and 20% BC5CDR,respectively.The model is better adapted to low resource named entity recognition tasks.

参考文献/References:

[1] HAMMERTON J.Named entity recognition with long short-term memory[EB/OL].[2022-04-03].http://dl.acm.org/ft_gateway.cfm?id=1119202&type=pdf.
[2] GUILLAUME L,Miguel B,SandeepS,et al.Neural architectures for named entity recognition[EB/OL].[2022-04-05].https://arxiv.org/pdf/1603.01360.pdf.
[3] HUANG Zhiheng,XU Wei,YU Kai.Bidirectional LSTM-CRF models for sequence tagging[EB/OL].[2022-04-02].https://arxiv.org/abs/1508.01991.
[4] 殷章志,李欣子,黄德根,等.融合字词模型的中文命名实体识别研究[J].中文信息学报,2019,33(11):95-100,106.
[5] 林广和,张绍武,林鸿飞.基于细粒度词表示的命名实体识别研究[J].中文信息学报,2018,32(11):62-71,78.
[6] RATHAPARKHI A.A maximum entropy part of speech tagger[EB/OL].[2022-04-06].https://www.mendeley.com/catalogue/51884aa1-bab9-3acc-8e85-51606f7087ce/.
[7] MCCALLUM A,FREITAG D,Pereira F C N.Maximum entropy Markov models for information extraction and segmentation[EB/OL].[2022-04-02].https://dl.acm.org/doi/10.5555/645529.658277.
[8] LAFFERTY J,MCCALLUM A,PEREIRA F C N.Conditional random fields:probabilistic models for segmenting and labeling sequence data[EB/OL].[2022-04-07].https://dl.acm.org/doi/10.5555/645530.655813.
[9] COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing(almost)from scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[10] CHIU J P C,NICHOLS E.Named entity recognition with bidirectional LSTM-CNNs[EB/OL].[2022-04-09].https://arxiv.org/pdf/1511.08308.pdf.
[11] NI Jian,GEORGIANA D,RADU F.Weakly supervised cross-lingual named entity recognition via effective annotation and representation projection[EB/OL].[2022-04-02].https://arxiv.org/abs/1707.02483.
[12] MAYHEW S,TSAI C T,ROTH D.Cheap translation for cross-lingual named entity recognition[EB/OL].[2022-04-05].https://aclanthology.org/D17-1268.pdf.
[13] TOOLKIT O S,Hc B,Ng M,et al.Moses:open source toolkit for statistical machine translation[EB/OL].[2022-04-02].https://dl.acm.org/doi/10.5555/1557769.1557821.
[14] CHEN Xilun,AHMED H A,HANYH,et al.Multi-source cross-lingual model transfer:learning what to share[EB/OL].[2022-04-04].https://aclanthology.org/P19-1299/.
[15] PHILLIP K,LU Yichao,VIKAS B.Adversarial learning with contextual embeddings for zero-resource cross-lingual classification and NER[EB/OL].[2022-04-02].https://arxiv.org/abs/1909.00153v3.
[16] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[EB/OL].[2022-04-02].http://de.arxiv.org/pdf/1406.2661.
[17] DAI Xiang,ADELH.An analysis of simple data augmentation for named entity recognition[EB/OL].[2022-04-04].https://arxiv.org/abs/2010.11683.
[18] CHEN Jiaao,WANG Zhenghui,TIAN Ran,et al.Local additivitybased data augmentation for semi-supervised NER[EB/OL].[2022-04-06].https://arxiv.org/abs/2010.01677.
[19] YANG Yaosheng,CHEN Wenliang,LI Zhenghua,et al.Distantly supervised NER with partial annotation learning and reinforcement learning[EB/OL].[2022-04-07].https://aclanthology.org/C18-1183.pdf.
[20] TSUBOI Y,KASHIMA H,MORI S,et al.Training conditional random fields using incomplete annotations[EB/OL].[2022-04-08].http://dl.acm.org/ft_gateway.cfm?id=1599194&type=pdf.
[21] FENG Jun,HUANG Minlie,ZHAO Li,et al.Reinforcement learning for relation classification from noisy data[EB/OL].[2022-04-04].https://arxiv.org/pdf/1808.08013.pdf.
[22] ZHANG Tao,XIA Congying,PHILIP S Y,et al.PDALN:progressive domain adaptation over a pre-trained model for low-resource cross-domain named entity recognition[EB/OL].[2022-04-04].https://aclanthology.org/2021.emnlp-main.442/.
[23] CHEN Siqi,PEI Yijie,KE Zunwang,et al.Low-resource named entity recognition via the pre-training model[J].Symmetry,2021,13(5):786.
[24] PENNINGTON J,SOCHER R,MANNING C D.Glove:global vectors for word representation[EB/OL].[2022-04-05].https://aclanthology.org/D14-1162/.
[25] JIE Zhanming,XIE Pengjun,LU Wei,et al.Better modeling of incomplete annotations for named entity recognition[EB/OL].[2022-04-02].https://www.xueshufan.com/publication/2945214158.
[26] ERIK F T,Fien D M.Introduction to the CoNLL-2003 shared task:language-independent named entity recognition[EB/OL].[2022-04-04].https://aclanthology.org/W03-0419/.
[27] LI Jiao,SUN Yueping,JOHNSON R J,et al.BioCreative V CDR task corpus:a resource for chemical disease relation extraction[EB/OL].[2022-04-02].https://pubmed.ncbi.nlm.nih.gov/27161011/.
[28] BILL Y L,LEE D H,SHEN Ming,et al.TriggerNER:learning with Entity triggers as explanations for Named Entity recognition[EB/OL].[2022-04-03].https://github.com/INK-USC/TriggerNER.
[29] ZHONG Maosheng,LIU Ganlin,XIONG Jian,et al.DualNER:atrigger based dual learning framework for low-resource Named Entity recognition[J].IEEE Intelligent Systems,2022,37(4):79-87.

相似文献/References:

[1]卢逸君,滕少华*.小样本数据生成及其在异常检测中的应用[J].江西师范大学学报(自然科学版),2020,(04):385.[doi:10.16357/j.cnki.issn1000-5862.2020.04.10]
　LU Yijun,TENG Shaohua*.The Generation of Minority Sample Data and Its Application in Abnormal Detection[J].Journal of Jiangxi Normal University:Natural Science Edition,2020,(05):385.[doi:10.16357/j.cnki.issn1000-5862.2020.04.10]
[2]邓 泓,刘志超,彭莹琼*,等.基于Fibonacci采样的数据预处理方法研究[J].江西师范大学学报(自然科学版),2021,(01):60.[doi:10.16357/j.cnki.issn1000-5862.2021.01.09]
　DENG Hong,LIU Zhichao,PENG Yingqiong*,et al.The Study on Data Preprocessing Method Based on Fibonacci Sampling[J].Journal of Jiangxi Normal University:Natural Science Edition,2021,(05):60.[doi:10.16357/j.cnki.issn1000-5862.2021.01.09]
[3]刘 磊,许 婕,周 勇*.基于知识增强的ERBERT-GRU中文图书分类方法研究[J].江西师范大学学报(自然科学版),2021,(03):299.[doi:10.16357/j.cnki.issn1000-5862.2021.03.12]
　LIU Lei,XU Jie,ZHOU Yong*.The Study on ERBERT-GRU Chinese Book Classification Method Based on Knowledge Enhancement[J].Journal of Jiangxi Normal University:Natural Science Edition,2021,(05):299.[doi:10.16357/j.cnki.issn1000-5862.2021.03.12]
[4]杨得国,马兰萍,聂 毓.基于PCANet和SVM的病变眼底图像检测算法[J].江西师范大学学报(自然科学版),2022,(04):372.[doi:10.16357/j.cnki.issn1000-5862.2022.04.07]
　YANG Deguo,MA Lanping,NIE Yu.The Detection Algorithm of Pathological Fundus Image Based on PCANet and SVM[J].Journal of Jiangxi Normal University:Natural Science Edition,2022,(05):372.[doi:10.16357/j.cnki.issn1000-5862.2022.04.07]

备注/Memo

备注/Memo:
收稿日期:2022-06-17
基金项目:国家自然科学基金(61877031)和江西省教育厅科技课题(GJJ210324)资助项目.
作者简介:钟茂生(1974—),男,江西兴国人,教授,博士,主要从事机器学习、自然语言处理和智能教育软件的研究.E-mail:zhongmaosheng@sina.com
更新日期/Last Update: 2022-09-25