融入BERT的企业年报命名实体识别方法

被引:14
作者
张靖宜 [1 ]
贺光辉 [1 ]
代洲 [2 ]
刘亚东 [1 ]
机构
[1] 上海交通大学电子信息与电气工程学院
[2] 南方电网物资有限公司
关键词
命名实体识别; 企业年报; BERT; 注意力机制; 双向门控循环单元;
D O I
10.16183/j.cnki.jsjtu.2020.009
中图分类号
TP391.1 [文字信息处理];
学科分类号
081203 ; 0835 ;
摘要
自动提取企业年报关键数据是企业评价工作自动化的重要手段.针对企业年报领域关键实体结构复杂、与上下文语义关联强、规模较小的特点,提出基于转换器的双向编码器表示-双向门控循环单元-注意力机制-条件随机场(BERT-BiGRU-Attention-CRF)模型.在BiGRU-CRF模型的基础上,首先引入BERT预训练语言模型,以增强词向量模型的泛化能力,捕捉长距离的上下文信息;然后引入注意力机制,以充分挖掘文本的全局和局部特征.在自行构建的企业年报语料库内进行实验,将该模型与多组传统模型进行对比.结果表明:该模型的F1值(精确率和召回率的调和平均数)为93.69%,对企业年报命名实体识别性能优于其他传统模型,有望成为企业评价工作自动化的有效方法.
引用
收藏
页码:117 / 123
页数:7
相关论文
共 13 条
[1]  
Phonologically aware neural model for named entity recognition in low resource transfer settings . BHARADWAJ A,MORTENSEN D,DYER C. Conference on Empirical Methods in Natural Language Processing . 2016
[2]  
Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism . CAO P F,CHEN Y B,LIU K,et al. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . 2018
[3]  
Attention is all you need . VASWANI A,SHAZEER N,PARMAR N,et al. Proceedings of the 31st Conference on Neural Information Processing Systems . 2017
[4]  
BERT:Pre-training of deep bidirectional transformers for language understanding[OL] . DEVLIN J,CHANG M W,LEE K,et al. https://arxiv.org/abs/ 1810.04805 . 2019
[5]  
Skip-thought vectors . KIROS R,ZHU Y K,SALAKHUTDINOV R R,et al. Proceedings of the 29th Conference on Neural Information Processing Systems . 2015
[6]  
Learning phrase representations using RNN encoder-decoder for statistical machine translation . Cho K,Merrienboer B V,Gulcehre C. Empirical Methods in Natural Language Processing . 2014
[7]  
End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF . Ma X,Hovy E. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. ACL . 2016
[8]  
Fast and accurate entity recognition with iterated dilated convolutions . Strubell E,Verga P,Belanger D. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing . 2017
[9]  
KEA:Practical Automatic Keyphrase Extraction . Witten Ian H G.P,Eibe Frank,et aI. Proceedings of the Fourth ACM Conference on Digital Libraries . 1999
[10]  
Joint extraction of entities and relations based on a novel tagging scheme . ZHENG S C,WANG F,BAO H Y,et al. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers) . 2017