基于层叠CRF模型的词结构分析

被引:6
作者
方艳 [1 ]
周国栋 [2 ]
机构
[1] 苏州大学自然语言处理实验室
[2] 苏州大学计算机科学与技术学院
关键词
中文分词; 内部结构; 分词标准; 层叠CRF;
D O I
暂无
中图分类号
TP391.1 [文字信息处理];
学科分类号
摘要
传统的中文分词就是识别出每个词的边界,它忽略了汉语中词与短语分界不清这一特点。在理论上,语言学家对词边界的确定往往各持己见,各语料库的分词标准不能统一,在实践中也不能完全满足具体应用的需求。该文给出了基于层叠CRF模型的词结构自动分析方法,能够以较高的精确度获得词的边界信息和内部结构信息。相比于传统的分词,词的结构分析更加符合汉语词法与句法边界模糊的事实,解决了语料库标准的不一致性以及应用的不同需求。
引用
收藏
页码:1 / 7+24 +24
页数:8
相关论文
共 13 条
[1]  
Automatic adaptation of annotation standards:Chinese word segmentation and POS tagging--a case study. Jiang Wenbin,Huang Liang,Liu Qun. Proceedings of theJoint Conference of the47th Annual Meeting of the ACL and the4th InternationalJoint Conference on Natural Language Processing of the AFNLP . 2009
[2]  
Word segmentation needschange-From a linguist’’s view. Dong Zhengdong,Dong Qiang,Hao Changling. Proceedings of CIPS-SIGHAN Joint Conference onChinese Language Processing . 2010
[3]  
Customizable segmentation of morphologically derived words inChinese. Wu Andi. Computational Linguistics . 2003
[4]  
Character-level dependencies in Chinese: Usefulness and learning. Zhao Hai. Proceedings of the12th Conference of the European Chapter of the ACL . 2009
[5]  
Adaptive Chinese word segmentation. Gao JF,Wu AD,Li M,Huang CN,Li HQ,Xia XS,Qin HW. Annual Meeting of the Association for Computational Linguistics . 2004
[6]   统计与词典相结合的领域自适应中文分词 [J].
张梅山 ;
邓知龙 ;
车万翔 ;
刘挺 .
中文信息学报, 2012, (02) :8-12
[7]   异种语料融合方法:基于统计的中文词法分析应用 [J].
孟凡东 ;
徐金安 ;
姜文斌 ;
刘群 .
中文信息学报, 2012, 26 (02) :3-7+12
[8]  
JointChinesewordsegmentation,POStaggingandparsing. QianXian,YangLiu. Proceedingsofthe2012JointConferenceonEmpiricalMethodsinNaturalLanguageProcessingandComputationalNaturalLanguageLearning . 2012
[9]  
An Improved Chinese WordSegmentation System with Conditional Random Field. Hai Zhao,Chang-Ning Huang,Mu Li. Proceedingsof the Fifth SIGHAN Workshop on Chinese Language Processing(SIGHAN-5) . 2006
[10]  
Fast Full Parsing by Linear-Chain Condi-tional Random Fields. Yoshimasa Tsuruoka,Jun’’ichi Tsujii,SophiaAnaniadou. Proceedings of the 12thConference of the European Chapter of the ACL (EACL 2009) . 2009