Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis

Evgeniy GabrilovichShaul Markovitch

article2007IJCAI2,396 citations

Shows how Explicit Semantic Analysis turns Wikipedia concepts into interpretable text vectors and substantially improves agreement with human semantic-relatedness judgments.

Listen

Automated systems frequently struggle to evaluate semantic relatedness—how closely two words or text passages relate in meaning—because computers lack the extensive world knowledge and common-sense context that humans naturally possess. Previous computational methods relied either on purely statistical word co-occurrence or on limited lexical dictionaries, both of which fail to capture broad domain knowledge and complex relationships between concepts.

The article sets out to introduce and evaluate Explicit Semantic Analysis, a method that leverages large-scale, human-curated knowledge repositories to compute the semantic relatedness of both individual words and unrestricted texts.

To accomplish this, the authors built a semantic interpreter using encyclopedic concepts from Wikipedia and directory categories from the Open Directory Project. The system indexes hundreds of thousands of human-defined concepts into a high-dimensional semantic space. Text classification algorithms map any input word or document into a weighted vector of these explicit concepts, after which standard vector comparison calculates their semantic closeness. The approach was evaluated against human judgments using standard benchmarks: the WordSimilarity-353 dataset containing 353 word pairs, and a collection of 50 news documents paired in 1,225 combinations.

The evaluation produced several critical findings. First, the Wikipedia-based approach substantially improved correlation with human judgments over existing state-of-the-art systems, increasing word-level correlation from 0.56 to 0.75 and document-level correlation from 0.60 to 0.72. Second, Wikipedia-based interpretation outperformed the Open Directory Project model across all tests, primarily because Wikipedia articles represent focused, orthogonal concepts with standard, high-quality language rather than noisy crawled web pages. Third, the system demonstrated natural word sense disambiguation by analyzing ambiguous terms within the surrounding context of neighboring words.

These findings indicate that large-scale, open knowledge repositories can effectively bridge the background-knowledge gap in computational linguistics. Because Explicit Semantic Analysis represents meaning through recognizable human concepts rather than abstract mathematical dimensions, its outputs are transparent, explainable, and practical for high-stakes applications such as information retrieval, text categorization, and intelligence analysis. The uniform framework also eliminates the need for separate models when analyzing isolated words versus extended text.

Organizations developing search, categorization, or text analysis capabilities should consider adopting concept-based semantic models over purely statistical or taxonomy-restricted approaches. System architects should favor clean, well-structured encyclopedic corpuses like Wikipedia over broad, uncurated web directories when building reference knowledge bases. The online interpretation speed of several hundred words per second supports operational integration, though teams should evaluate throughput requirements against their specific scale.

Confidence in these results is supported by strong correlation against established human benchmarks. However, leaders should note that the approach requires an initial offline index-building stage and depends on the ongoing quality and coverage of the underlying reference encyclopedia. Domain-specific applications with highly specialized terminology outside general encyclopedic coverage may require supplementary custom knowledge sources.

Technion – Israel Institute of Technology semantic information semantic distance Semantic Relatedness Concepts

Gabrilovich et al (2007).pdf

Paper: Indexing By Latent Semantic Analysis, Scott Deerwester et al. (1990). Latent Semantic Indexing laid the foundational low-dimensional vector approach for automated information retrieval that Explicit Semantic Analysis extends using explicit knowledge bases.
Paper: Unsupervised Learning by Probabilistic Latent Semantic Analysis, Thomas Hofmann (2001). Probabilistic Latent Semantic Analysis established generative mixture models for term-document co-occurrence, offering the statistical modeling background that explicit concept mapping builds upon.

Paper: GloVe: Global Vectors for Word Representation, Jeffrey Pennington et al. (2014). GloVe extends the exploration of vector space models by combining global matrix factorization with co-occurrence statistics to capture finer-grained semantic relationships.
Paper: Efficient Estimation of Word Representations in Vector Space, Tomáš Mikolov et al. (2013). Mikolov et al. continue the evolution of semantic word representations by introducing highly efficient continuous vector architectures like skip-gram and CBOW.
Paper: Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks, Nils Reimers et al. (2019). Sentence-BERT builds directly on static and contextual word embedding foundations to produce efficient sentence-level semantic representations using siamese networks.
Paper: Learning deep structured semantic models for web search using clickthrough data, Po-Sen Huang et al. (2013). Deep Structured Semantic Models advance semantic matching for search by applying deep neural networks to learn representations from user clickthrough data.

Cover for Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis

Abstract

Computing semantic relatedness of natural language texts requires access to vast amounts of common-sense and domain-specific world knowledge. We propose Explicit Semantic Analysis (ESA), a novel method that represents the meaning of texts in a high-dimensional space of concepts derived from Wikipedia. We use machine learning techniques to explicitly represent the meaning of any text as a weighted vector of Wikipedia-based concepts. Assessing the relatedness of texts in this space amounts to comparing the corresponding vectors using conventional metrics (e.g., cosine). Compared with the previous state of the art, using ESA results in substantial improvements in correlation of computed relatedness scores with human judgments: from r = 0.56 to 0.75 for individual words and from r = 0.60 to 0.72 for texts. Importantly, due to the use of natural concepts, the ESA model is easy to explain to human users.

Topics

semantic information semantic distance

Keywords

1 Introduction
2 Explicit Semantic Analysis
3 Empirical Evaluation
3.1 Datasets and Evaluation Procedure
3.2 Results
4 Related Work
5 Conclusions
6 Acknowledgments
References

Knowls

Knowl 1 — Explicit Semantic Analysis Text Representation
model/method
Explicit Semantic Analysis (ESA) represents the semantic content of unrestricted natural language text in a high-dimensional space of human-defined concepts derived from knowledge repositories such as Wikipedia or the Open Directory Project (ODP). In ESA, each concept corresponds to an encyclopedia article or directory category (e.g., Computer Science, India). Each concept $c_j$ is represented as an attribute vector of words weighted by term frequency-inverse document frequency (TF-IDF), quantifying the strength of association between words and that concept.

Unlike Latent Semantic Analysis (LSA), which reduces dimensionality to uninterpretable latent dimensions, ESA maps text into explicitly interpretable concepts grounded in human knowledge. To enable efficient online inference, an inverted index is constructed mapping each word to the list of concepts in which it appears along with its association weight, discarding low-weight associations below a significance threshold.
Knowl 2 — Semantic Interpretation Vector and Relatedness Computation
equation
Let an input text fragment $T = \{w_i\}$ be represented as a TF-IDF vector $v$ , where $v_i$ is the weight of word $w_i \in T$ . Let $k_{ij}$ denote the inverted index association weight (TF-IDF strength) between word $w_i$ and Wikipedia concept $c_j \in \{c_1, \dots, c_N\}$ , where $N$ is the total number of concepts.

The semantic interpretation vector $V(T) \in \mathbb{R}^N$ for text $T$ has its $j$ -th component $V_j$ defined as the centroid classifier score:

$V_j = \sum_{w_i \in T} v_i \cdot k_{ij}$

The semantic relatedness between two text fragments $T_1$ and $T_2$ is computed as the cosine similarity between their normalized semantic interpretation vectors $V(T_1)$ and $V(T_2)$ :

$\text{Relatedness}(T_1, T_2) = \frac{V(T_1) \cdot V(T_2)}{\|V(T_1)\| \|V(T_2)\|}$
Knowl 3 — Explicit Semantic Analysis Interpretation Algorithm
algorithm
The ESA semantic interpreter translates an arbitrary text fragment into a weighted concept interpretation vector by accumulating concept weights from an inverted index.
Input: Natural language text fragment $T$ , inverted index $I$ mapping words to concept-weight pairs $(c_j, k_{ij})$ , total concept count $N$
Output: Semantic interpretation vector $V \in \mathbb{R}^N$
$V \leftarrow$ zero vector of dimension $N$
Compute TF-IDF weight vector $v$ for all unique words $w_i \in T$
for each word $w_i \in T$ do
     $v_i \leftarrow$ TF-IDF weight of $w_i$ in $T$
    Retrieve postings list $L(w_i) = \{(c_j, k_{ij})\}$ from inverted index $I$
    for each $(c_j, k_{ij}) \in L(w_i)$ do
         $V_j \leftarrow V_j + v_i \cdot k_{ij}$
    end for
end for
return $V$
The resulting vector $V$ represents the text fragment as a weighted mixture of explicit Wikipedia concepts ordered by relevance.

Knowl 4 — Word Relatedness Evaluation on WordSimilarity-353

data/table

Semantic relatedness between individual words was evaluated on the WordSimilarity-353 dataset (353 word pairs, each scored by 13–16 human annotators). Algorithm scores were compared with averaged human ratings using the Spearman rank-order correlation coefficient ( $r$ ).

ESA-Wikipedia achieves a correlation of $r = 0.75$ , substantially improving over lexical taxonomy methods (WordNet: 0.33–0.35; Roget's Thesaurus: 0.55), statistical co-occurrence models (LSA: 0.56), and Wikipedia title-link path methods (WikiRelate!: 0.19–0.48).

Knowl 5 — Document Semantic Relatedness Evaluation on the ABC News Corpus

data/table

Document-level semantic relatedness was evaluated on 50 documents from the Australian Broadcasting Corporation news mail service paired in all 1,225 combinations, with 8–12 human judgments per pair. Because averaging human judgments produced 67 distinct values, Pearson's linear correlation coefficient ( $r$ ) was used for evaluation.

ESA-Wikipedia achieves $r = 0.72$ , outperforming both the standard bag-of-words approach ( $r = 0.10\text{--}0.50$ ) and LSA ( $r = 0.60$ ).

Knowl 6 — Knowledge Base Construction and Preprocessing for ESA
experimental setup
The ESA semantic interpreter was instantiated on two large-scale knowledge repositories:
1. Wikipedia: An XML snapshot from March 26, 2006 contained 2.9 GB of text in 1,187,839 articles. Overly specific or short articles (fewer than 100 words and fewer than 5 incoming or outgoing links) were pruned, leaving 241,393 concept articles. After removing stop words and rare words and applying stemming, 389,202 distinct terms were retained as attribute vector dimensions.
2. Open Directory Project (ODP): An April 2004 snapshot pruned of non-English branches contained over 400,000 category concepts and 2,800,000 URLs (436 MB text). Crawling the first 10 pages of each cataloged site added 70 GB of text. After stop-word and rare-word elimination, 20,700,000 distinct terms were obtained. The top 1,000 most informative terms per ODP node were selected via document frequency to build centroid concept vectors.
Knowl 7 — Conceptual Orthogonality and Text Quality in Wikipedia versus ODP for ESA
empirical result
Across both word-level ( $r = 0.75$ vs. $0.65$ ) and document-level ( $r = 0.72$ vs. $0.69$ ) relatedness benchmarks, Wikipedia-based ESA outperforms ODP-based ESA. Two main factors drive this difference:
1. Concept Orthogonality: The axes of an interpretation space should ideally be as orthogonal as possible. Wikipedia articles each focus on a discrete, standalone topic. In contrast, ODP is organized as a hierarchical taxonomy whose parent-child generalization links directly violate concept orthogonality.
2. Text Quality and Noise: Wikipedia articles provide high-quality Standard Written English with minimal noise. In contrast, expanding ODP concept nodes by crawling external web pages incorporates substantial unstructured noise inherent to arbitrary web pages.
Knowl 8 — Context-Driven Word Sense Disambiguation in ESA
model/method
Because ESA constructs interpretation vectors by summing concept associations across all words in a multi-word phrase or passage, surrounding words provide context that automatically performs word sense disambiguation.

When an ambiguous term appears alongside other words, concepts corresponding to the intended sense receive cumulative weight from all related context words, whereas concepts representing irrelevant senses receive minimal aggregate score. For instance:
- The phrase "Bank of America" produces top concepts related to finance (Bank, Bank of America, MBNA, VISA (credit card), NASDAQ).
- The phrase "Bank of Amazon" produces top concepts related to river geography (Amazon River, Amazon Basin, Amazon Rainforest, Atlantic Ocean, Brazil).
- The phrase "Jaguar car models" produces automobile concepts (Jaguar (car), Jaguar S-Type, Daimler, Luxury vehicles).
- The phrase "Jaguar (Panthera onca)" produces feline concepts (Jaguar, Felidae, Black panther, Leopard, Puma, Tiger).
Knowl 9 — Semantic Relatedness versus Semantic Similarity Across Lexical and Encyclopedic Models
empirical result
On narrowly defined semantic similarity benchmarks measuring classical taxonomic relations (e.g., Rubenstein & Goodenough with 65 word pairs and Miller & Charles with 30 word pairs), lexical resources perform well, yielding human correlations of $r = 0.70\text{--}0.85$ , compared to ESA's $r = 0.816$ on R&G and $r = 0.723$ on M&C.

However, when evaluated on general semantic relatedness (which includes non-taxonomic associations such as meronymy, antonymy, and functional relationships on WordSimilarity-353), WordNet-based measures drop sharply to $r = 0.33\text{--}0.35$ . In contrast, ESA achieves $r = 0.75$ , demonstrating that encyclopedic concept models capture broader multi-faceted semantic associations than strict taxonomic lexical networks.

Coverage note — None was omitted; all primary contributions, mathematical equations, algorithms, empirical benchmarks, and qualitative analyses from the paper are covered.

References

1.Ricardo Baeza-Yates and Berthier Ribeiro-Neto. Modern Information Retrieval. Addison Wesley, New York, NY, 1999.
2.Satanjeev Banerjee and Ted Pedersen. Extended gloss overlaps as a measure of semantic relatedness. In IJCAI, pages 805–810, 2003.
3.B. G. Buchanan and E. A. Feigenbaum. Forward. In R. Davis and D. B. Lenat, editors, Knowledge-Based Systems in Artificial Intelligence. McGraw-Hill, 1982.
4.Alexander Budanitsky and Graeme Hirst. Evaluating wordnet-based measures of lexical semantic relatedness. Computational Linguistics, 32(1):13–47, 2006.
5.Ido Dagan, Lillian Lee, and Fernando C. N. Pereira. Similarity-based models of word cooccurrence probabilities. Machine Learning, 34(1–3):43–69, 1999.
6.S. Deerwester, S. Dumais, G. Furnas, T. Landauer, and R. Harshman. Indexing by latent semantic analysis. JASIS, 41(6):391–407, 1990.
7.Christiane Fellbaum, editor. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA, 1998.
8.Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. Placing search in context: The concept revisited. ACM TOIS, 20(1):116–131, January 2002.
9.Evgeniy Gabrilovich and Shaul Markovitch. Feature generation for text categorization using world knowledge. In IJCAI'05, pages 1048–1053, 2005.
10.Evgeniy Gabrilovich and Shaul Markovitch. Overcoming the brittleness bottleneck using Wikipedia: Enhancing text categorization with encyclopedic knowledge. In AAAI'06, pages 1301–1306, July 2006.
11.Evgeniy Gabrilovich. Feature Generation for Textual Information Retrieval Using World Knowledge. PhD thesis, Department of Computer Science, Technion—Israel Institute of Technology, Haifa, Israel, In preparation.
12.Jim Giles. Internet encyclopaedias go head to head. Nature, 438:900–901, 2005.
13.Gregory Grefenstette. SEXTANT: Exploring unexplored contexts for semantic extraction from syntactic analysis. In ACL'92, pages 324–326, 1992.
14.Eui-Hong (Sam) Han and George Karypis. Centroid-based document classification: Analysis and experimental results. In PKDD'00, September 2000.
15.Mario Jarmasz. Roget's thesaurus as a lexical resource for natural language processing. Master's thesis, University of Ottawa, 2003.
16.Jay J. Jiang and David W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In ROCLING'97, 1997.
17.Michael D. Lee, Brandon Pincombe, and Matthew Welsh. An empirical evaluation of models of text document similarity. In CogSci2005, pages 1254–1259, 2005.
18.Lillian Lee. Measures of distributional similarity. In Proceedings of the 37th Annual Meeting of the ACL, 1999.
19.D. Lenat and R. Guha. Building Large Knowledge Based Systems. Addison Wesley, 1990.
20.Dekang Lin. An information-theoretic definition of word similarity. In ICML'98, 1998.
21.Rada Mihalcea, Courtney Corley, and Carlo Strapparava. Corpus-based and knowledge-based measures of text semantic similarity. In AAAI'06, July 2006.
22.George A. Miller and Walter G. Charles. Contextual correlates of semantic similarity. Language and Cognitive Processes, 6(1):1–28, 1991.
23.Philip Resnik. Semantic similarity in a taxonomy: An information-based measure and its application to problems of ambiguity in natural language. JAIR, 11:95–130, 1999.
24.Peter Roget. Roget's Thesaurus of English Words and Phrases. Longman Group Ltd., 1852.
25.Herbert Rubenstein and John B. Goodenough. Contextual correlates of synonymy. Communications of the ACM, 8(10):627–633, 1965.
26.Mehran Sahami and Timothy Heilman. A web-based kernel function for measuring the similarity of short text snippets. In WWW'06. ACM Press, May 2006.
27.G. Salton and M.J. McGill. An Introduction to Modern Information Retrieval. McGraw-Hill, 1983.
28.Fabrizio Sebastiani. Machine learning in automated text categorization. ACM Comp. Surv., 34(1):1–47, 2002.
29.Michael Strube and Simon Paolo Ponzetto. WikiRelate! Computing semantic relatedness using Wikipedia. In AAAI'06, Boston, MA, 2006.
30.Justin Zobel and Alistair Moffat. Exploring the similarity space. ACM SIGIR Forum, 32(1):18–34, 1998.

Citation

MLA

Gabrilovich, E., and S. Markovitch. “Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis”. International Joint Conference on Artificial Intelligence, 2007, pp. 1606–11, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.76.9790.

APA

Gabrilovich, E., & Markovitch, S. (2007). Computing semantic relatedness using Wikipedia-based explicit semantic analysis. International Joint Conference on Artificial Intelligence, 1606–1611. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.76.9790

Chicago

Gabrilovich, E., and S. Markovitch. 2007. “Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis”. International Joint Conference on Artificial Intelligence, 1606–11. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.76.9790.

Harvard

Gabrilovich, E. and Markovitch, S. (2007) “Computing semantic relatedness using Wikipedia-based explicit semantic analysis”, International Joint Conference on Artificial Intelligence, pp. 1606–1611. Available at: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.76.9790.

Vancouver

1. Gabrilovich E, Markovitch S (2007) Computing semantic relatedness using Wikipedia-based explicit semantic analysis. International Joint Conference on Artificial Intelligence 1606–1611

BibTeX

@article{gabrilovich2007computing,
  title = {Computing semantic relatedness using Wikipedia-based explicit semantic analysis},
  author = {Gabrilovich, Evgeniy and Markovitch, Shaul},
  year = {2007},
  journal = {International Joint Conference on Artificial Intelligence},
  pages = {1606-1611},
  url = {http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.76.9790}
}

Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Algorithm	Correlation with humans ( $r$ )
WordNet	0.33–0.35
Roget's Thesaurus	0.55
Latent Semantic Analysis (LSA)	0.56
WikiRelate!	0.19–0.48
ESA-ODP	0.65
ESA-Wikipedia	0.75

Algorithm	Correlation with humans ( $r$ )
Bag of words	0.10–0.50
Latent Semantic Analysis (LSA)	0.60
ESA-ODP	0.69
ESA-Wikipedia	0.72

Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Explicit Semantic Analysis Text Representation

Knowl 2 — Semantic Interpretation Vector and Relatedness Computation

Knowl 3 — Explicit Semantic Analysis Interpretation Algorithm

Knowl 4 — Word Relatedness Evaluation on WordSimilarity-353

Knowl 5 — Document Semantic Relatedness Evaluation on the ABC News Corpus

Knowl 6 — Knowledge Base Construction and Preprocessing for ESA

Knowl 7 — Conceptual Orthogonality and Text Quality in Wikipedia versus ODP for ESA

Knowl 8 — Context-Driven Word Sense Disambiguation in ESA

Knowl 9 — Semantic Relatedness versus Semantic Similarity Across Lexical and Encyclopedic Models

References

Citation

BibTeX

Access the Paper