Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

word-context pairs

Word-context pairs are paired data units in computational linguistics and natural language processing that consist of a target word and an associated context element, typically another word appearing within a predefined neighboring window of text or a specific syntactic relationship in a corpus. Extracted by scanning text corpora, these pairs record co-occurrence instances to quantify the distributional patterns of language. They serve as the foundational data for both count-based distributional models, such as pointwise mutual information matrices, and prediction-based neural word embeddings, such as the skip-gram architecture, enabling computational systems to represent lexical meaning and semantic similarity in vector spaces based on the principle that words occurring in similar contexts share similar meanings.

1 item

Improving Distributional Similarity with Lessons Learned from Word Embeddings

Improving Distributional Similarity with Lessons Learned from Word Embeddings

Omer Levy, Yoav Goldberg, Ido Dagan

OrganizationsBar-Ilan University

Why you should read this

Demonstrates that the superior performance of neural word embeddings over traditional count-based models stems from hyperparameter optimizations rather than algorithmic differences, proving that applying these same tuning strategies to count-based methods eliminates the performance gap across semantic benchmarks.

Recent trends suggest that neural-network-inspired word embedding models outperform traditional count-based distributional models on word similarity and analogy detection tasks. We reveal that much of the performance gains of word embeddings are due to certain system design choices and hyperparameter optimizations, rather than the embedding algorithms themselves. Furthermore, we show that these modifications can be transferred to traditional distributional models, yielding similar gains. In contrast to prior reports, we observe mostly local or insignificant performance differences between the methods, with no global advantage to any single approach over the others.

Added

2026-09-25