[1]刘 磊,许 婕,周 勇*.基于知识增强的ERBERT-GRU中文图书分类方法研究[J].江西师范大学学报(自然科学版),2021,(03):299-304.[doi:10.16357/j.cnki.issn1000-5862.2021.03.12]
　LIU Lei,XU Jie,ZHOU Yong*.The Study on ERBERT-GRU Chinese Book Classification Method Based on Knowledge Enhancement[J].Journal of Jiangxi Normal University:Natural Science Edition,2021,(03):299-304.[doi:10.16357/j.cnki.issn1000-5862.2021.03.12]
点击复制

基于知识增强的ERBERT-GRU中文图书分类方法研究()
分享到：

《江西师范大学学报》（自然科学版）[ISSN:1006-6977/CN:61-1281/TN]

卷:
期数:
2021年03期
页码:
299-304
栏目:
信息科学与技术
出版日期:
2021-06-10

文章信息/Info

Title:
The Study on ERBERT-GRU Chinese Book Classification Method Based on Knowledge Enhancement
文章编号:
1000-5862(2021)03-0299-06
作者:
刘 磊1; 许 婕2; 周 勇1*
1. 江西师范大学计算机信息工程学院,江西 南昌 330022; 2.江西师范大学图书馆,江西 南昌 330022
Author(s):
LIU Lei1; XU Jie2; ZHOU Yong1*
1.College of Computer and Information Engineering,Jiangxi Normal University,Nanchang Jiangxi 330022,China; 2.Libray,Jiangxi Normal University,Nanchang Jiangxi 330022,China
关键词:
图书分类;  ERBERT-GRU模型;  神经网络;  深度学习;  知识嵌入
Keywords:
book classification;  ERBERT-GRU model;  neural network;  deep learning;  knowledge embedding
分类号:
TP 311
DOI:
10.16357/j.cnki.issn1000-5862.2021.03.12
文献标志码:
A
摘要:
图书的自动分类是图书管理和图书推荐算法中的基础工作,也是难点之一,而且目前针对中文分类算法主要集中在短文本领域中,鲜有对图书等长文本分类的研究.该文对深度学习分类算法进行了深入细致的研究,并对BERT预训练模型及其变体进行相应的改进.利用复杂层级网络叠加双向Transformer编码器来提取隐藏在文本中的细粒度信息.在预训练过程中,增加实体级别的遮罩,获得对传统BERT模型的改进,提高了模型对中文语义理解的能力.通过添加外部知识提升了该模型的鲁棒性.
Abstract:
The automatic classification of books is the basic work in book management and book recommendation algorithms,and it is also one of the difficulties.At present,the Chinese classification algorithms are mainly concentrated in the field of short texts,and there are few studies on the classification of long texts such as books.The in-depth and detailed study on deep learning classification algorithms is conducted,mainly studying the BERT pre-training model and its variants and making corresponding improvements.A complex hierarchical network is used to superimpose a two-way transformer encoder to extract the fine-grained information hidden in the text.By adding an entity-level mask in the pre-training process,the traditional BERT model is improved,and the model is improved in Chinese semantic comprehension.The ability to understand semantics improves the robustness of the model by adding external knowledge.

参考文献/References:

[1] Devlin J,Chang Mingwei,Lee K,et al.BERT:pre-training of deep bidirectional transformers for language understanding[J].Computation and Language,2018,26(1):4171-4186.
[2] Kim Y.Convolutional neural networks for sentence classification[J].Eprint Arxiv,2014,14(10):1746-1751.
[3] Liu Pengfei,Qiu Xipeng,Huang Xuanjing.Recurrent neural network for text classification with multi-task learning[EB/OL].[2016-06-17].https://arxiv.org/abs/1605.05101.
[4] Malhotra P,Vig L,Shroff G,et al.Long short term memory networks for anomaly detection in time series[EB/OL].[2019-01-19]. https://www.researchgate.net/publication/304782562_Long_Short_Term_Memory_Networks_for_Anomaly_Detection_in_Time_Series.
[5] Cho K,Van M B,Gulcehre C,et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[EB/OL].[2019-01-19].https://arxiv.org/abs/1406.1078v3.
[6] Chen Zhenguo,Shi Guang,Wang Xiaoju.Text classification based on naive Bayes algorithm with feature selection[J].International Journal on Information,2012,24(10):4255-4260.
[7] Joachims T.Text categorization with support vector machines:learning with many relevant features[C].Berlin:Springer,1998:127-142.
[8] Vries A D,Mamoulis N,Nes N,et al.Efficient KNN searchonverticallydecomposeddata[C].Madiso:ACM Press,2002:322-333.
[9] Mikolov T.Distributed representations of words and phrases and their compositionality[J].Advances in Neural Information Processing Systems,2013,2(26):3111-3119.
[10] Bengio Y,Ducharme R,Vincent P,et al.A neural probabilistic language model[J].Journal of Machine Learning Research,2003,3(2):1137-1155.
[11] Pennington J,Socher R,Manning C.Glove:global vectors for word representation[C].Doha:Association for Computational Linguistics,2014:1532-1543.
[12] Peters M E,Neumann M,Iyyer M,et al.Deep contextualized word representations[EB/OL].[2019-01-19].https://www.aclweb.org/anthology/N18-1202.pdf.
[13] Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need[EB/OL].[2019-01-19].https://arxiv.org/abs/1706.03762v5.
[14] Mikolov T,Martin Karafiát,Burget L,et al.Recurrent neural network based language model[C].Chiba:DBLP,2015:1045-1048.
[15] Kingma D P,Ba J.Adam:a method for stochastic optimization[EB/OL].[2019-01-19].https://arxiv.org/abs/1412.6980v9.

备注/Memo

备注/Memo:
收稿日期:2020-09-16
基金项目:江西省教育厅科学技术研究(KJLD14021)和江西省教育厅省重点教改课题(JXJG1821)资助项目.
通信作者:周 勇(1971—),男,江西南昌人,副研究员,主要从事数据库、数据挖掘和人工智能方面的研究.E-mail:zhou_yong@126.com
更新日期/Last Update: 2021-06-10