Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

second-order similarity

Second-order similarity is a metric in computational linguistics and distributional semantics that measures the degree of similarity between two words or concepts based on the overlap and resemblance of their respective context distributions, rather than direct co-occurrence with one another. While first-order similarity quantifies how frequently two words appear alongside each other to reflect associative or syntagmatic relationships, second-order similarity evaluates paradigmatic relationships by determining how interchangeable two items are across similar environments. Consequently, two terms exhibiting high second-order similarity share similar patterns of association with other words throughout a text corpus and map close to each other in vector spaces or word embedding models, even if they never directly appear together in the same sentence.

1 item

Improving Distributional Similarity with Lessons Learned from Word Embeddings

Improving Distributional Similarity with Lessons Learned from Word Embeddings

Omer Levy, Yoav Goldberg, Ido Dagan

OrganizationsBar-Ilan University

Why you should read this

Demonstrates that the superior performance of neural word embeddings over traditional count-based models stems from hyperparameter optimizations rather than algorithmic differences, proving that applying these same tuning strategies to count-based methods eliminates the performance gap across semantic benchmarks.

Recent trends suggest that neural-network-inspired word embedding models outperform traditional count-based distributional models on word similarity and analogy detection tasks. We reveal that much of the performance gains of word embeddings are due to certain system design choices and hyperparameter optimizations, rather than the embedding algorithms themselves. Furthermore, we show that these modifications can be transferred to traditional distributional models, yielding similar gains. In contrast to prior reports, we observe mostly local or insignificant performance differences between the methods, with no global advantage to any single approach over the others.

Added

2026-09-25