Dependency-Based Word Embeddings

被引:542
作者
Levy, Omer [1 ]
Goldberg, Yoav [1 ]
机构
[1] Bar Ilan Univ, Comp Sci Dept, Ramat Gan, Israel
来源
PROCEEDINGS OF THE 52ND ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 2 | 2014年
关键词
D O I
10.3115/v1/p14-2050
中图分类号
TP39 [计算机的应用];
学科分类号
080201 [机械制造及其自动化];
摘要
While continuous word embeddings are gaining popularity, current models are based solely on linear contexts. In this work, we generalize the skip-gram model with negative sampling introduced by Mikolov et al. to include arbitrary contexts. In particular, we perform experiments with dependency-based contexts, and show that they produce markedly different embeddings. The dependency-based embeddings are less topical and exhibit more functional similarity than the original skip-gram embeddings.
引用
收藏
页码:302 / 308
页数:7
相关论文
共 32 条
[1]
Al-Rfou Rami, 2013, P CONLL 2013
[2]
[Anonymous], 2009, N AM CHAPTER ASS COM
[3]
[Anonymous], 2009, Advances in neural information processing systems
[4]
[Anonymous], 2008, COLING 2008 P WORKSH
[5]
[Anonymous], 2014, CoRR, abs/1402.3722.
[6]
Distributional Memory: A General Framework for Corpus-Based Semantics [J].
Baroni, Marco ;
Lenci, Alessandro .
COMPUTATIONAL LINGUISTICS, 2010, 36 (04) :673-721
[7]
Bengio Y, 2001, ADV NEUR IN, V13, P932
[8]
Brown Peter F., 1992, COMPUT LINGUIST, V18, P4
[9]
Extracting semantic representations from word co-occurrence statistics: A computational study [J].
Bullinaria, John A. ;
Levy, Joseph P. .
BEHAVIOR RESEARCH METHODS, 2007, 39 (03) :510-526
[10]
SEMANTIC AND ASSOCIATIVE PRIMING IN THE CEREBRAL HEMISPHERES - SOME WORDS DO, SOME WORDS DONT ... SOMETIMES, SOME PLACES [J].
CHIARELLO, C ;
BURGESS, C ;
RICHARDS, L ;
POLLOCK, A .
BRAIN AND LANGUAGE, 1990, 38 (01) :75-104