Unsupervised Learning by Probabilistic Latent Semantic Analysis

Thomas Hofmann

article2001Machine Learning2,738 citations

Introduces Probabilistic Latent Semantic Analysis (PLSA), a generative latent variable framework trained with tempered Expectation-Maximization that overcomes the geometric limitations of standard LSA to effectively model polysemy and substantially improve document retrieval and language modeling performance.

Listen

The article addresses the challenge of automatically processing large volumes of text to support applications such as search, filtering, and language understanding, driven by the growth of the World Wide Web. Existing Latent Semantic Analysis (LSA) methods map word counts to lower-dimensional spaces but rest on an ad hoc linear-algebra foundation that struggles with polysemous words and lacks a clear statistical basis.

The article set out to develop and test Probabilistic Latent Semantic Analysis (PLSA), a generative latent-class model that decomposes co-occurrence data through probabilistic mixture components rather than singular-value decomposition.

Researchers formulated an aspect model that associates each word occurrence with a hidden context variable, then fitted the model to term-document matrices using the Expectation-Maximization algorithm and a tempered variant that controls overfitting. They evaluated predictive performance via perplexity on two text collections and measured retrieval quality on four standard information-retrieval test sets containing between roughly one and three thousand documents each.

PLSA reduced perplexity by factors of approximately 3.3 on the MED collection and 2.4 on noun-adjective pairs, compared with factors of 1.9 and 2.1 for LSA. In document retrieval, PLSA raised average precision by 17 to 58 percent over a term-frequency baseline across the four collections, while LSA produced smaller gains or none at all. Combining multiple PLSA models further improved results. Tempered fitting consistently outperformed standard early-stopping EM.

These gains indicate that a statistically grounded model yields more accurate representations of word usage and document similarity, directly improving search relevance and language-model quality at modest extra computational cost.

Organizations should consider replacing or augmenting LSA-based indexing with PLSA for collections where precision matters, and they should apply tempered fitting to guard against overfitting. Further work is needed to test the approach on larger, more diverse corpora and to integrate it with existing retrieval pipelines.

The main limitations are that EM can reach only local maxima and that performance still depends on choosing an appropriate number of latent classes and temperature schedule; results were obtained on medium-sized, English-language collections, so generalization to other domains or languages requires additional validation. Overall the experiments provide strong evidence that PLSA outperforms LSA on the reported tasks.

Brown University inference latent factor latent variable Probabilistic Latent Semantic Analysis Expectation Maximization Latent Class Model

Hofmann (2001).pdf

Paper: Indexing By Latent Semantic Analysis, Scott Deerwester et al. (1990). It introduces Latent Semantic Analysis via singular-value decomposition, establishing the exact dimensionality-reduction paradigm and retrieval limitations that PLSA is designed to overcome.
Paper: Probabilistic latent semantic indexing, Thomas Hofmann (1999). It details the foundational probabilistic aspect model and expectation-maximization framework for text indexing that the source article directly extends and evaluates.
Paper: Probabilistic Latent Semantic Analysis, Thomas Hofmann (1999). It establishes the core statistical formulation of Probabilistic Latent Semantic Analysis and tempered expectation-maximization on co-occurrence data.
Paper: Text Classification from Labeled and Unlabeled Documents using EM, K. Nigam et al. (2000). It provides the foundational mechanics of utilizing the Expectation-Maximization algorithm for generative text modeling.
Paper: Term-weighting approaches in automatic text retrieval, Gerard Salton et al. (1988). It establishes standard term-weighting schemes and vector-space retrieval baselines evaluated against latent topic models.

Paper: Latent Dirichlet Allocation, David M. Blei et al. (2003). It directly builds upon and generalizes PLSA into a fully generative Bayesian model by placing Dirichlet priors over document-topic distributions.
Paper: Reading Tea Leaves: How Humans Interpret Topic Models, Jonathan D. Chang et al. (2009). It evaluates the human interpretability and semantic coherence of topic representations produced by models including pLSI and LDA.
Paper: Exploring the Space of Topic Coherence Measures, Michael Röder et al. (2015). It systematically explores automated topic coherence metrics to quantify the quality of latent semantic representations learned by topic models.
Paper: From Frequency to Meaning: Vector Space Models of Semantics, Peter D. Turney et al. (2010). It provides a comprehensive survey of vector space models of semantics, contextualizing latent semantic and probabilistic decomposition techniques within broader NLP paradigms.
Paper: A Neural Probabilistic Language Model, Yoshua Bengio et al. (2003). It extends statistical language modeling beyond discrete latent class mixtures to continuous distributed vector representations trained with neural networks.

Cover for Unsupervised Learning by Probabilistic Latent Semantic Analysis

Abstract

This paper presents a novel statistical method for factor analysis of binary and count data which is closely related to a technique known as Latent Semantic Analysis. In contrast to the latter method which stems from linear algebra and performs a Singular Value Decomposition of co-occurrence tables, the proposed technique uses a generative latent class model to perform a probabilistic mixture decomposition. This results in a more principled approach with a solid foundation in statistical inference. More precisely, we propose to make use of a temperature controlled version of the Expectation Maximization algorithm for model fitting, which has shown excellent performance in practice. Probabilistic Latent Semantic Analysis has many applications, most prominently in information retrieval, natural language processing, machine learning from text, and in related areas. The paper presents perplexity results for different types of text and linguistic data collections and discusses an application in automated document indexing. The experiments indicate substantial and consistent improvements of the probabilistic method over standard Latent Semantic Analysis.

Topics

inference latent factor latent variable semantic information supervised machine learning statistical methods unsupervised learning method syntactics natural language understanding

Keywords

Probabilistic Latent Semantic Analysis Expectation Maximization Latent Class Model Mixture Model Dimension Reduction Information Retrieval Natural Language Processing Language Modeling Tempered EM Perplexity

2. Latent semantic analysis
2.1. Count data and co-occurrence tables
2.2. Latent semantic analysis by singular value decomposition
3. Probabilistic latent semantic analysis
3.1. The aspect model
3.2. Model fitting with the EM algorithm
3.3. Latent probability spaces and probabilistic latent semantic analysis
3.4. Intermezzo: Word usage analysis with the aspect model
3.5. Aspects versus clusters
3.6. Model fitting revisited: Improving generalization by tempered EM
4. Experimental results
4.1. Perplexity evaluation for PLSA and LSA
4.2. Information retrieval with PLSA and LSA
5. Conclusion
Acknowledgment
References

Knowls

Knowl 1 — Probabilistic Latent Semantic Analysis Generative Model
model/method
Probabilistic Latent Semantic Analysis (PLSA), also known as the aspect model, is a latent variable model for discrete dyadic count data, such as a collection of text documents $\mathcal{D} = \{d_1, \dots, d_N\}$ over a vocabulary $\mathcal{W} = \{w_1, \dots, w_M\}$ . Each observed word-document co-occurrence pair $(d_i, w_j)$ is associated with an unobserved latent class variable $z_k \in \{z_1, \dots, z_K\}$ .

In the asymmetric parameterization, data generation proceeds by:
1. Selecting a document $d_i$ with prior probability $P(d_i)$ .
2. Selecting a latent class (topic or aspect) $z_k$ with document-conditional probability $P(z_k | d_i)$ .
3. Emitting a word $w_j$ with class-conditional probability $P(w_j | z_k)$ .
The resulting joint probability mass function is: $P(d_i, w_j) = P(d_i) P(w_j | d_i) = P(d_i) \sum_{k=1}^K P(w_j | z_k) P(z_k | d_i)$

Reversing the conditional dependency yields the equivalent symmetric parameterization: $P(d_i, w_j) = \sum_{k=1}^K P(z_k) P(d_i | z_k) P(w_j | z_k)$

The model assumes conditional independence between document $d_i$ and word $w_j$ given the latent variable $z_k$ . Parameters are fitted by maximizing the log-likelihood of the observed term counts $n(d_i, w_j)$ : $\mathcal{L} = \sum_{i=1}^N \sum_{j=1}^M n(d_i, w_j) \log P(d_i, w_j) = \sum_{i=1}^N n(d_i) \left[ \log P(d_i) + \sum_{j=1}^M \frac{n(d_i, w_j)}{n(d_i)} \log \sum_{k=1}^K P(w_j | z_k) P(z_k | d_i) \right]$ where $n(d_i) = \sum_{j=1}^M n(d_i, w_j)$ denotes the document length and $P(d_i) \propto n(d_i)$ .
Knowl 2 — Expectation-Maximization Algorithm for PLSA
algorithm
Model parameters of PLSA are fitted by the Expectation-Maximization (EM) algorithm. The algorithm alternates between an E-step, which computes the posterior distribution over latent classes $z_k$ for every observed document-word pair $(d_i, w_j)$ , and an M-step, which re-estimates the conditional probability distributions $P(w_j | z_k)$ and $P(z_k | d_i)$ by maximizing the expected complete-data log-likelihood subject to probability simplex normalization constraints.
Input: Term-document co-occurrence counts $n(d_i, w_j)$ for $N$ documents and $M$ words, number of latent classes $K$
Output: Fitted probability distributions $P(w_j | z_k)$ and $P(z_k | d_i)$
Initialize $P(w_j | z_k)$ and $P(z_k | d_i)$ randomly such that $\sum_{j=1}^M P(w_j | z_k) = 1$ and $\sum_{k=1}^K P(z_k | d_i) = 1$
repeat
    // E-step: compute posterior over latent classes for all non-zero pairs
    for each observed pair $(d_i, w_j)$ with $n(d_i, w_j) > 0$ do
        for $k = 1$ to $K$ do
             $P(z_k | d_i, w_j) \leftarrow \frac{P(w_j | z_k) P(z_k | d_i)}{\sum_{l=1}^K P(w_j | z_l) P(z_l | d_i)}$
        end for
    end for
    // M-step: re-estimate aspect-word and document-aspect distributions
    for $k = 1$ to $K$ do
        for $j = 1$ to $M$ do
             $P(w_j | z_k) \leftarrow \frac{\sum_{i=1}^N n(d_i, w_j) P(z_k | d_i, w_j)}{\sum_{m=1}^M \sum_{i=1}^N n(d_i, w_m) P(z_k | d_i, w_m)}$
        end for
    end for
    for $i = 1$ to $N$ do
         $n(d_i) \leftarrow \sum_{j=1}^M n(d_i, w_j)$
        for $k = 1$ to $K$ do
             $P(z_k | d_i) \leftarrow \frac{\sum_{j=1}^M n(d_i, w_j) P(z_k | d_i, w_j)}{n(d_i)}$
        end for
    end for
until convergence criterion or early stopping threshold is reached
Each EM iteration requires $\mathcal{O}(R \cdot K)$ arithmetic operations, where $R$ is the number of non-zero entries in the term-document matrix and $K$ is the number of latent classes. Typically, 20 to 50 iterations are sufficient for model fitting.
Knowl 3 — Tempered Expectation Maximization Algorithm
algorithm
Tempered Expectation Maximization (TEM) regularizes the EM estimation of PLSA models by introducing an entropic penalty derived from minimizing the generalized Helmholtz free energy: $\mathcal{F}_\beta = -\beta \sum_{i=1}^N \sum_{j=1}^M n(d_i, w_j) \sum_{k=1}^K \tilde{P}(z_k; d_i, w_j) \log \left[ P(d_i | z_k) P(w_j | z_k) P(z_k) \right] + \sum_{i=1}^N \sum_{j=1}^M n(d_i, w_j) \sum_{k=1}^K \tilde{P}(z_k; d_i, w_j) \log \tilde{P}(z_k; d_i, w_j)$ where $\beta \in (0, 1]$ represents the inverse computational temperature and $\tilde{P}(z_k; d_i, w_j)$ are variational posterior distributions.

Minimizing $\mathcal{F}_\beta$ w.r.t. $\tilde{P}$ yields the tempered posterior: $\tilde{P}(z_k; d_i, w_j) = \frac{\left[ P(z_k | d_i) P(w_j | z_k) \right]^\beta}{\sum_{l=1}^K \left[ P(z_l | d_i) P(w_j | z_l) \right]^\beta}$ Lower values of $\beta < 1$ smooth the posterior probabilities toward the uniform distribution, countering overfitting. Rather than traditional slow forward annealing, an inverse annealing strategy is used:
Input: Term-document counts $n(d_i, w_j)$ , hold-out validation set, temperature step factor $\eta \in (0, 1)$
Output: Regularized PLSA parameters $P(w_j | z_k)$ and $P(z_k | d_i)$
Set $\beta \leftarrow 1.0$
Perform standard EM iterations with early stopping based on hold-out perplexity
repeat
     $\beta \leftarrow \eta \cdot \beta$
    Perform one TEM iteration using tempered posteriors $\tilde{P}(z_k; d_i, w_j)$
    while validation performance on hold-out data improves do
        Perform another TEM iteration at the current $\beta$
    end while
until decreasing $\beta$ yields no further performance improvement on hold-out data
Knowl 4 — Geometric and Algebraic Comparison Between PLSA and LSA
theoretical result
Geometrically, the $K$ class-conditional probability mass functions $P(\cdot | z_k)$ over vocabulary $\mathcal{W}$ form points on the $(M-1)$ -dimensional probability simplex. Their convex hull defines a $(K-1)$ -dimensional convex region $\mathcal{R} = \mathrm{conv}(P(\cdot | z_1), \dots, P(\cdot | z_K))$ within the simplex. Each document distribution $P(\cdot | d_i)$ is approximated by a convex combination within $\mathcal{R}$ , with mixing weights $P(z_k | d_i)$ acting as continuous latent coordinates.

Algebraically, both Latent Semantic Analysis (LSA) and PLSA decompose dyadic relations into low-rank matrix products:
- LSA computes a Singular Value Decomposition (SVD) of count matrix $N \approx U \Sigma V^T$ , optimizing the Frobenius ( $L_2$ ) norm. This corresponds to an implicit additive Gaussian noise assumption on count variables, which can yield negative entries in reconstructed matrices, does not produce normalized probability distributions, and provides no clear probabilistic interpretation of singular vectors.
- PLSA expresses the joint probability matrix as $P = \hat{U} \hat{\Sigma} \hat{V}^T$ , where $\hat{U} = (P(d_i | z_k))_{i,k}$ , $\hat{V} = (P(w_j | z_k))_{j,k}$ , and $\hat{\Sigma} = \mathrm{diag}(P(z_k))$ . PLSA optimizes the multinomial sampling log-likelihood (minimizing Kullback-Leibler divergence between empirical and model distributions), guaranteeing non-negative, properly normalized probability distributions where latent dimensions correspond directly to interpretable class-conditional word distributions (topics).
Knowl 5 — Probabilistic Latent Semantic Indexing and Folding-In Mechanism
model/method
Probabilistic Latent Semantic Indexing (PLSI) applies PLSA representations to information retrieval by mapping documents and queries into the latent factor space $P(z_k | d_i)$ and $P(z_k | q)$ .

To compute representations for new queries $q$ (or unseen test documents) without retraining the model, a folding-in procedure is executed:
1. The trained aspect-word conditional probabilities $P(w_j | z_k)$ are held constant.
2. The query mixing coefficients $P(z_k | q)$ are computed via EM/TEM iterations that optimize the likelihood of the query words $n(q, w_j)$ under the fixed conditional models: $P(z_k | q) \propto \sum_{j=1}^M n(q, w_j) \tilde{P}(z_k | q, w_j)$ where $\tilde{P}(z_k | q, w_j) = \frac{[P(z_k | q) P(w_j | z_k)]^\beta}{\sum_l [P(z_l | q) P(w_j | z_l)]^\beta}$ .
Similarity between a document $d_i$ and a query $q$ is computed by taking the cosine between their latent factor vectors: $s_{\mathrm{latent}}(d_i, q) = \frac{\sum_{k=1}^K P(z_k | d_i) P(z_k | q)}{\sqrt{\sum_{k=1}^K P(z_k | d_i)^2} \sqrt{\sum_{k=1}^K P(z_k | q)^2}}$ To combine semantic abstraction with exact lexical matching, the final score combines $s_{\mathrm{latent}}(d_i, q)$ with standard term-matching cosine similarity $s_{\mathrm{term}}(d_i, q)$ : $s(d_i, q) = \lambda s_{\mathrm{term}}(d_i, q) + (1 - \lambda) s_{\mathrm{latent}}(d_i, q)$ where $\lambda \in [0, 1]$ is a combination weight parameter.
Knowl 6 — Theoretical Distinction Between the Aspect Model and Document Clustering Models
theoretical result
In probabilistic document clustering models (such as multinomial AutoClass or distributional clustering), each document $d_i$ as a whole is assumed to belong to exactly one latent cluster $c_k$ . The document-conditional word probability is given by: $P(w_j | d_i) = \sum_{k=1}^K P(c(d_i) = c_k) P(w_j | c_k)$ where $P(c(d_i) = c_k) \propto P(c_k) \prod_{j=1}^M P(w_j | c_k)^{n(d_i, w_j)}$ . As document length $n(d_i) = \sum_j n(d_i, w_j)$ increases, the posterior $P(c(d_i) = c_k)$ rapidly concentrates all probability mass onto a single cluster $c_k$ , forcing the document to inherit only that cluster's word distribution.

In contrast, the PLSA aspect model associates a latent variable $z_k$ with each individual word occurrence $(d_i, w_j)$ rather than the entire document. Although word tokens in $d_i$ share document-specific priors $P(z_k | d_i)$ , distinct word occurrences within the same document can be generated by different aspects. Consequently, PLSA models documents as continuous convex combinations of topics and explicitly models within-document polysemy.
Knowl 7 — Uniform Model Averaging in Probabilistic Latent Semantic Indexing
model/method
In Latent Semantic Analysis, SVD solutions for different latent dimensionalities $K$ form a nested sequence of subspaces. In contrast, independent PLSA models trained with different random initializations or varying topic dimensions $K$ converge to distinct local likelihood maxima that capture different semantic aspects.

The combined retrieval method, denoted $\mathrm{PLSI}^*$ , pools an ensemble $S$ of independently trained PLSA models (e.g., trained with $K \in \{32, 48, 64, 80, 128\}$ ) by computing the unweighted uniform average of their latent cosine similarities: $s_{\mathrm{PLSI}^*}(d_i, q) = \lambda s_{\mathrm{term}}(d_i, q) + (1 - \lambda) \frac{1}{|S|} \sum_{m \in S} s_{\mathrm{latent}}^{(m)}(d_i, q)$ where $s_{\mathrm{term}}(d_i, q)$ is the standard term frequency cosine score and $\lambda \in [0, 1]$ . Model averaging reduces estimation noise and provides high empirical stability with respect to the choice of $\lambda$ and individual model weights.
Knowl 8 — Perplexity Metric for Statistical Language Models on Dyadic Data
definition
Generalization performance of a statistical co-occurrence model on an unseen hold-out or test dataset $\mathcal{D}'$ is evaluated using perplexity $\mathcal{P}$ , defined as the exponential of the negative log-averaged conditional probability of words given documents: $\mathcal{P} = \exp \left[ -\frac{\sum_{i=1}^N \sum_{j=1}^M n'(d_i, w_j) \log P(w_j | d_i)}{\sum_{i=1}^N \sum_{j=1}^M n'(d_i, w_j)} \right]$ where $n'(d_i, w_j)$ denotes the occurrence count of word $w_j$ in document $d_i$ in the test partition, and $P(w_j | d_i) = \sum_{k=1}^K P(w_j | z_k) P(z_k | d_i)$ is the predicted conditional distribution. Lower perplexity reflects higher predictive likelihood on unseen data.

Knowl 9 — Average Precision Comparison of LSI, PLSI, and Combined PLSI* on Document Collections

data/table

Information retrieval performance was evaluated across four standard medium-sized benchmark collections with expert ground truth: MED (1033 medical abstracts), CRAN (1400 aeronautics abstracts), CACM (3204 computer science abstracts), and CISI (1460 library science abstracts). The baseline method is cosine similarity on raw term-frequency vectors ( $\mathrm{cos} + \mathrm{tf}$ ). Average precision across 9 recall levels ( $10\%$ to $90\%$ ) and relative improvements over baseline are summarized below.

	MED		CRAN		CACM		CISI
Method	prec.	impr.	prec.	impr.	prec.	impr.	prec.	impr.
cos + tf	44.3	–	29.9	–	17.9	–	12.7	–
LSI	51.7	+16.7%	*28.7	-4.0%	*16.0	-11.6%	12.8	+0.8%
PLSI	63.9	+44.2%	35.1	+17.4%	22.9	+27.9%	18.8	+48.0%
PLSI*	66.3	+49.7%	37.5	+25.4%	26.8	+49.7%	20.1	+58.3%

Asterisks indicate that LSI failed to improve over the baseline (reporting results at 256 dimensions with combination weight $\lambda = 2/3$ ). While LSI causes negative performance gains on CRAN ( $-4.0\%$ ) and CACM ( $-11.6\%$ ), single PLSI achieves substantial gains on all datasets ( $+17.4\%$ to $+48.0\%$ ). Uniform model combination ( $\mathrm{PLSI}^*$ ) consistently achieves the highest precision, with improvements reaching $+58.3\%$ over the term-matching baseline.

Knowl 10 — Empirical Perplexity Evaluation of PLSA, LSA, and TEM Regularization
empirical result
Predictive perplexity was evaluated as a function of latent space dimensionality $K$ on the MED document collection ( $N=1033$ , matrix rank 1033) and on noun-adjective pairs from the LOB corpus (matrix rank 1674, predicting nouns conditioned on adjectives):
1. On the MED dataset (unigram baseline perplexity 3073), standard LSA achieves a minimum perplexity of 1647 ( $1.87\times$ reduction), whereas PLSA trained with Tempered EM (TEM) achieves a perplexity of 936 ( $3.28\times$ reduction).
2. On the LOB dataset (unigram baseline perplexity 1316), LSA reduces perplexity to 632 ( $2.08\times$ reduction), while PLSA with TEM reduces perplexity to 547 ( $2.41\times$ reduction).
3. On the MED dataset, TEM model fitting significantly outperforms standard early-stopping EM at all dimensions $K$ . Furthermore, TEM allows training models where the number of latent classes exceeds the matrix rank (tested up to $K=2048$ ) with continuous reductions in hold-out perplexity.

Coverage note — None. All major theoretical formulations, algorithms (EM, TEM), geometric interpretations, comparisons with LSA and clustering, indexing methods (PLSI, PLSI*), and empirical evaluations (perplexity and IR benchmarks) are covered.

References

1.Baker, L. D. & McCallum, A. K. (1998). Distributional clustering of words for text classification. In Proceedings of the 21st ACM-SIGIR International Conference on Research and Development in Information Retrieval (SIGIR).
2.Bellegarda, J. R. (1998). Exploiting both local and global constraints for multi-span statistical language modeling. In Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’98, pp. 677–680.
3.Berry, M. W. Dumais, S. T., & O’Brien, G. W. (1995). Using linear algebra for intelligent information retrieval. SIAM Review, 37(4), 573–595.
4.Cheeseman, P. & Stutz, J. (1996). Bayesian classification (AutoClass): Theory and results. In Usama M. Fayyad, Gregory Piatetsky-Shapiro, Padhraic Smyth, & Ramasamy Uthurusamy, (Eds.), Advances in Knowledge Discovery and Data Mining. AAAI Press/MIT Press.
5.Coccaro, N. & Jurafsky, D. (1998). Towards better integration of semantic predictors in statistical language modeling. In Proceedings of the 5th International Conference on Spoken Language Processing (ICSLP).
6.Deerwester, S., Dumais, G. W., Furnas, S. T., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41, 391–407.
7.Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. J. Royal Statist. Soc. B, 39, 1–38.
8.Dumais, S. T. (1995). Latent semantic indexing (LSI): TREC-3 report. In D.K Harman, (Ed.), Proceedings of the Text REtrieval Conference (TREC-3), pp. 219–230.
9.Foltz, P. W. & Dumais, S. T. (1992). An analysis of information filtering methods. Communications of the ACM, 35(12), 51–60.
10.Gilula, Z., & Haberman, S. J. (1986). Canonical analysis of contingency tables by maximum likelihood. Journal of the American Statistical Association, 81(395), 780–788.
11.Golub, G. H. & Van Loan, C. F. (1996). Matrix Computations. Johns Hopkins University Press, 3rd (ed.).
12.Hofmann, T., Puzicha, J., & Jordan, M. I. (1999). Unsupervised learning from dyadic data. In Advances in Neural Information Processing Systems, Vol. 11, MIT Press.
13.Katz, S. M. (1987). Estimation of probabilities for sparse data for the language model component of a speech recogniser. IEEE Transactions on Acoustics, Speech and Signal Processing, 35(3), 400–401.
14.Landauer, T. K. & Dumais, S. T. (1997). A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review.
15.LDC. Linguistic Data Consortium: TDT pilot study corpus documentation. http://www.ldc.upenn.edu/TDT, 1997.
16.Lee, D. D. & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401(675), 788–791.
17.Neal, R. M. & Hinton, G. E. (1998). A view of the EM algorithm that justifies incremental and other variants. In M.I. Jordan, (Ed.), Learning in Graphical Models, Dordrecht, MA: Kluwer Academic Publishers, pp. 355–368.
18.Pereira, F. C. N., Tishby, N. Z., & Lee, L. (1983). Distributional clustering of english words. In Proceedings of the ACL, pp. 183–190.
19.Rose, K., Gurewitz, E., & Fox, G. (1990). A deterministic annealing approach to clustering. Pattern Recognition Letters, 11(11), 589–594.
20.Salton, G. & McGill, M. J. (1983). Introduction to Modern Information Retrieval. New York: McGraw–Hill.
21.Saul, L. & Pereira, F. (1997). Aggregate and mixed–order Markov models for statistical language processing. In Proceedings of the 2nd International Conference on Empirical Methods in Natural Language Processing, pp. 81–89.
22.Ueda, N. & Nakano, R. (1988). Deterministic annealing EM algorithm. Neural Networks, 11(2), 271–282.
23.Witten, I. H. & Bell, T. C. (1991). The zero-frequency problem—estimating the probabilities of novel events in adaptive text compression. IEEE Transactions on Information Theory, 37(4);1085–1094.
24.Wolfe, M. B. W., Schreiner, M. E., Rehder, B., Laham, D., Foltz, P. W., Kintsch, W. & Landauer, T. K. (1998). Learning from text: Matching readers and texts by latent semantic analysis. Discourse Processes, 25(2/3), 309–336.

Citation

MLA

Hofmann, T. “Unsupervised Learning by Probabilistic Latent Semantic Analysis”. Machine Learning, vol. 42, nos. 1-2, 2001, pp. 177–96, https://doi.org/10.1023/A:1007617005950.

APA

Hofmann, T. (2001). Unsupervised Learning by Probabilistic Latent Semantic Analysis. Machine Learning, 42(1-2), 177–196. https://doi.org/10.1023/A:1007617005950

Chicago

Hofmann, T. 2001. “Unsupervised Learning by Probabilistic Latent Semantic Analysis”. Machine Learning 42 (1-2): 177–96. https://doi.org/10.1023/A:1007617005950.

Harvard

Hofmann, T. (2001) “Unsupervised Learning by Probabilistic Latent Semantic Analysis”, Machine Learning, 42(1-2), pp. 177–196. Available at: https://doi.org/10.1023/A:1007617005950.

Vancouver

1. Hofmann T (2001) Unsupervised Learning by Probabilistic Latent Semantic Analysis. Machine Learning 42:177–196

BibTeX

@article{Hofmann_2001, title={Unsupervised Learning by Probabilistic Latent Semantic Analysis}, volume={42}, ISSN={1573-0565}, url={http://dx.doi.org/10.1023/A:1007617005950}, DOI={10.1023/a:1007617005950}, number={1-2}, journal={Machine Learning}, publisher={Springer Science and Business Media LLC}, author={Hofmann, Thomas}, year={2001}, month=Jan, pages={177–196} }

Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Unsupervised Learning by Probabilistic Latent Semantic Analysis

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Probabilistic Latent Semantic Analysis Generative Model

Knowl 2 — Expectation-Maximization Algorithm for PLSA

Knowl 3 — Tempered Expectation Maximization Algorithm

Knowl 4 — Geometric and Algebraic Comparison Between PLSA and LSA

Knowl 5 — Probabilistic Latent Semantic Indexing and Folding-In Mechanism

Knowl 6 — Theoretical Distinction Between the Aspect Model and Document Clustering Models

Knowl 7 — Uniform Model Averaging in Probabilistic Latent Semantic Indexing

Knowl 8 — Perplexity Metric for Statistical Language Models on Dyadic Data

Knowl 9 — Average Precision Comparison of LSI, PLSI, and Combined PLSI* on Document Collections

Knowl 10 — Empirical Perplexity Evaluation of PLSA, LSA, and TEM Regularization

References

Citation

BibTeX

Access the Paper