Co-regularized Multi-view Spectral Clustering

Abhishek KumarPiyush RaiHal Daumé

article2011NeurIPS1,327 citations

Proposes a multi-view spectral clustering framework that co-regularizes graph Laplacians across different data representations to find consistent cluster assignments across diverse views.

Listen

Modern data analysis frequently involves complex datasets represented across multiple distinct sources or feature sets, known as views. Examples include webpages described by both text and hyperlinks, or documents translated into multiple languages. While each view can be analyzed independently, single-source analysis often discards valuable complementary relationships. Standard clustering methods fail to effectively combine these sources in an unsupervised manner without clear labels, creating an operational challenge for organizations seeking unified patterns across multi-source data.

The article develops and evaluates an unsupervised spectral clustering framework that simultaneously leverages multiple views by enforcing agreement across view-specific groupings. It demonstrates how co-regularization—traditionally used in semi-supervised learning—can be adapted to discover consistent underlying cluster structures across diverse representations.

The authors introduce two mathematical formulations: a pairwise scheme that aligns the similarity matrices of individual view representations, and a centroid-based scheme that aligns each view to a shared consensus representation. The approach was evaluated on two synthetic benchmarks and three real-world datasets spanning multilingual text, handwritten digits, and multi-feature image recognition. The methods were benchmarked against standard single-view baselines and conventional fusion techniques, including feature concatenation, kernel addition, kernel multiplication, and canonical correlation analysis.

The primary finding is that both co-regularized approaches consistently outperform or match all baseline methods across all tested datasets. For example, on multilingual document categorization, the pairwise method achieved a normalized mutual information score of 0.375, markedly exceeding the best single-view baseline (0.287) and standard fusion techniques like kernel product (0.123). On the image dataset, where standard fusion baselines actually degraded performance below the single-view level (0.510), the pairwise approach steadily improved as more views were integrated, reaching a score of 0.564 with four views. Additionally, the algorithms proved computationally practical, converging reliably in fewer than 10 iterations across a wide range of regularization weights.

These results demonstrate that naive data-combination strategies, such as concatenating features or simply multiplying kernels, can introduce noise and degrade clustering accuracy compared to analyzing a single view. The proposed framework reduces performance risks by adaptively updating the combined representations during optimization. This allows organizations to effectively integrate multi-modal data streams without requiring expensive manual annotations.

Organizations handling multi-view datasets should consider co-regularized spectral clustering over ad hoc feature concatenation, especially when data modalities differ significantly. When using the centroid-based formulation, practitioners should assign lower weighting parameters to suspected noisy views to protect the consensus representation. Future implementation work should focus on establishing formal theoretical convergence bounds and testing extensions for handling missing values across views.

University of Maryland University of Utah clustering methods data points multiple views co-regularized multi-view spectral clustering spectral clustering consensus clustering

Kumar et al (2011).pdf

Paper: A tutorial on spectral clustering, Ulrike von Luxburg (2007). This tutorial provides the foundational theory and graph Laplacian formulations for standard spectral clustering that the source extends to multi-view co-regularization.
Paper: On Spectral Clustering: Analysis and an algorithm, Andrew Y. Ng et al. (2001). This seminal paper introduces normalized spectral clustering on affinity graphs, establishing the core single-view spectral clustering algorithm adapted by the source.
Paper: Kernel k-means: spectral clustering and normalized cuts, Inderjit S. Dhillon et al. (2004). It establishes the theoretical connection between graph cuts, kernel k-means, and spectral clustering trace optimizations used to formulate the co-regularization objectives.
Paper: Self-Tuning Spectral Clustering, Lihi Zelnik-Manor et al. (2004). It analyzes eigenvector alignment and graph construction principles in spectral clustering that motivate consistent cross-view spectral embeddings.
Paper: Cluster Ensembles – A Knowledge Reuse Framework for Combining Multiple Partitions, Alexander Strehl et al. (2002). It introduces foundational consensus frameworks for combining multi-partition and multi-view clustering information that the source formulates via continuous spectral co-regularization.

Paper: Deep Canonical Correlation Analysis, Galen Andrew et al. (2013). It advances multi-view representation learning from spectral graph co-regularization to non-linear deep canonical correlation analysis.
Paper: Contrastive Multi-View Representation Learning on Graphs, Kaveh Hassani et al. (2020). It modernizes multi-view consistency on graph-structured data by employing contrastive neural representations across multiple graph views.
Paper: VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Adrien Bardes et al. (2021). It extends multi-view cross-regularization concepts to self-supervised deep joint-embedding architectures via explicit variance, invariance, and covariance penalties.
Paper: Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss, Jeff Z. HaoChen et al. (2021). It provides theoretical guarantees linking spectral decomposition on multi-view augmentation graphs to contrastive representation learning.

Cover for Co-regularized Multi-view Spectral Clustering

Abstract

In many clustering problems, we have access to multiple views of the data each of which could be individually used for clustering. Exploiting information from multiple views, one can hope to find a clustering that is more accurate than the ones obtained using the individual views. Often these different views admit same underlying clustering of the data, so we can approach this problem by looking for clusterings that are consistent across the views, i.e., corresponding data points in each view should have same cluster membership. We propose a spectral clustering framework that achieves this goal by co-regularizing the clustering hypotheses, and propose two co-regularization schemes to accomplish this. Experimental comparisons with a number of baselines on two synthetic and three real-world datasets establish the efficacy of our proposed approaches.

Topics

clustering methods data points multiple views regularization spectral clustering viewpoint hier-archical clustering data clustering

Keywords

co-regularized multi-view spectral clustering spectral clustering consensus clustering Multiple Kernel Learning Canonical Correlation Analysis pair-wise co-regularizers view-specific eigenvectors similarity matrices Frobenius norm

1 Introduction
2 Co-regularized Spectral Clustering
2.1 Pairwise Co-regularization
2.2 Extension to Multiple Views
2.3 Centroid-Based Co-regularization
3 Experiments
3.1 Results
4 Related Work
5 Discussion
References

Knowls

Knowl 1 — Pairwise Co-regularized Multi-view Spectral Clustering
model/method
Pairwise co-regularized multi-view spectral clustering combines multiple views of a dataset by encouraging the spectral embedding matrices of different views to agree pairwise. Let $n$ be the number of data points, $k$ be the target number of clusters, and $m$ be the number of distinct views. For each view $v \in \{1, \dots, m\}$ , let $K^{(v)} \in \mathbb{R}^{n \times n}$ denote the similarity (kernel) matrix, $D^{(v)} \in \mathbb{R}^{n \times n}$ the diagonal degree matrix with $D_{ii}^{(v)} = \sum_{j=1}^n K_{ij}^{(v)}$ , and $\mathcal{L}^{(v)} = (D^{(v)})^{-1/2} K^{(v)} (D^{(v)})^{-1/2}$ the normalized graph Laplacian.

Let $U^{(v)} \in \mathbb{R}^{n \times k}$ be the orthogonal spectral embedding matrix for view $v$ . The joint optimization objective across all $m$ views is:

$\max_{U^{(1)}, \dots, U^{(m)} \in \mathbb{R}^{n \times k}} \sum_{v=1}^m \text{tr}\left( (U^{(v)})^T \mathcal{L}^{(v)} U^{(v)} \right) + \lambda \sum_{1 \le v, w \le m, v \ne w} \text{tr}\left( U^{(v)} (U^{(v)})^T U^{(w)} (U^{(w)})^T \right) \quad \text{s.t.} \quad (U^{(v)})^T U^{(v)} = I_k, \; \forall 1 \le v \le m$

where $\lambda > 0$ is a trade-off parameter controlling agreement between view pairs and $I_k$ is the $k \times k$ identity matrix. For two views ( $m=2$ ), the objective simplifies to:

$\max_{U^{(1)}, U^{(2)} \in \mathbb{R}^{n \times k}} \text{tr}\left( (U^{(1)})^T \mathcal{L}^{(1)} U^{(1)} \right) + \text{tr}\left( (U^{(2)})^T \mathcal{L}^{(2)} U^{(2)} \right) + \lambda \text{tr}\left( U^{(1)} (U^{(1)})^T U^{(2)} (U^{(2)})^T \right) \quad \text{s.t.} \quad (U^{(1)})^T U^{(1)} = I_k, \; (U^{(2)})^T U^{(2)} = I_k$

When optimizing for view $v$ with all other view embeddings fixed, the subproblem reduces to standard spectral clustering with an adaptively modified Laplacian:

$\max_{U^{(v)} \in \mathbb{R}^{n \times k}} \text{tr}\left( (U^{(v)})^T \left( \mathcal{L}^{(v)} + \lambda \sum_{w=1, w \ne v}^m U^{(w)} (U^{(w)})^T \right) U^{(v)} \right) \quad \text{s.t.} \quad (U^{(v)})^T U^{(v)} = I_k$

The closed-form global optimum for each subproblem is given by the top- $k$ eigenvectors of $\mathcal{L}^{(v)} + \lambda \sum_{w \ne v} U^{(w)} (U^{(w)})^T$ .
Knowl 2 — Centroid-Based Co-regularized Multi-view Spectral Clustering
model/method
Centroid-based co-regularized multi-view spectral clustering regularizes the view-specific spectral embedding matrices towards a shared consensus centroid embedding matrix $U^* \in \mathbb{R}^{n \times k}$ . For $m$ views on $n$ data points clustered into $k$ clusters, let $\mathcal{L}^{(v)} = (D^{(v)})^{-1/2} K^{(v)} (D^{(v)})^{-1/2} \in \mathbb{R}^{n \times n}$ be the normalized graph Laplacian of view $v$ , $U^{(v)} \in \mathbb{R}^{n \times k}$ be the orthogonal embedding matrix of view $v$ , and $\lambda_v > 0$ be a view-specific regularization weight reflecting the reliability or informativeness of view $v$ .

The joint optimization problem is:

$\max_{U^{(1)}, \dots, U^{(m)}, U^* \in \mathbb{R}^{n \times k}} \sum_{v=1}^m \text{tr}\left( (U^{(v)})^T \mathcal{L}^{(v)} U^{(v)} \right) + \sum_{v=1}^m \lambda_v \text{tr}\left( U^{(v)} (U^{(v)})^T U^* (U^*)^T \right) \quad \text{s.t.} \quad (U^{(v)})^T U^{(v)} = I_k, \; \forall 1 \le v \le m, \quad (U^*)^T U^* = I_k$

This formulation scales with $m$ regularization terms rather than the $\binom{m}{2}$ terms required by pairwise co-regularization. In an alternating maximization framework, the conditional update rules are:
1. For view embedding $U^{(v)}$ with $U^*$ and all $U^{(w)}$ ( $w \ne v$ ) fixed:
$\max_{U^{(v)} \in \mathbb{R}^{n \times k}} \text{tr}\left( (U^{(v)})^T \left( \mathcal{L}^{(v)} + \lambda_v U^* (U^*)^T \right) U^{(v)} \right) \quad \text{s.t.} \quad (U^{(v)})^T U^{(v)} = I_k$

which is solved by setting $U^{(v)}$ to the top- $k$ eigenvectors of $\mathcal{L}^{(v)} + \lambda_v U^* (U^*)^T$ .
1. For the consensus centroid $U^*$ with all view-specific embeddings $\{U^{(v)}\}_{v=1}^m$ fixed:
$\max_{U^* \in \mathbb{R}^{n \times k}} \text{tr}\left( (U^*)^T \left( \sum_{v=1}^m \lambda_v U^{(v)} (U^{(v)})^T \right) U^* \right) \quad \text{s.t.} \quad (U^*)^T U^* = I_k$

which is solved by setting $U^*$ to the top- $k$ eigenvectors of $\sum_{v=1}^m \lambda_v U^{(v)} (U^{(v)})^T$ . The final clustering is obtained directly by applying $k$ -means to the rows of $U^*$ .
Knowl 3 — Spectral Clustering Hypothesis Disagreement Metric
definition
In multi-view spectral clustering, let $U^{(v)} \in \mathbb{R}^{n \times k}$ and $U^{(w)} \in \mathbb{R}^{n \times k}$ be the orthogonal eigenvector embedding matrices for views $v$ and $w$ , where $n$ is the number of data points, $k$ is the number of clusters, and $(U^{(v)})^T U^{(v)} = I_k$ . Under a linear kernel on the embedding space, the pairwise similarity matrix for view $v$ is $K_{U^{(v)}} = U^{(v)} (U^{(v)})^T$ .

The disagreement between the clustering hypotheses of two views is measured by the squared Frobenius norm of the difference between their normalized similarity matrices:

$D(U^{(v)}, U^{(w)}) = \left\| \frac{K_{U^{(v)}}}{\|K_{U^{(v)}}\|_F^2} - \frac{K_{U^{(w)}}}{\|K_{U^{(w)}}\|_F^2} \right\|_F^2$

Since $\|K_{U^{(v)}}\|_F^2 = \text{tr}(U^{(v)} (U^{(v)})^T U^{(v)} (U^{(v)})^T) = \text{tr}(I_k) = k$ for any orthonormal $U^{(v)}$ , the denominators are identical and constant across all valid orthogonal embeddings. Expanding the squared Frobenius norm and removing constants yields the simplified disagreement measure:

$D(U^{(v)}, U^{(w)}) = -\text{tr}\left( U^{(v)} (U^{(v)})^T U^{(w)} (U^{(w)})^T \right)$

Minimizing this disagreement between views is therefore mathematically equivalent to maximizing $\text{tr}\left( U^{(v)} (U^{(v)})^T U^{(w)} (U^{(w)})^T \right)$ .
Knowl 4 — Alternating Maximization Algorithm for Pairwise Co-regularized Spectral Clustering
algorithm
The pairwise co-regularized multi-view spectral clustering algorithm optimizes the joint spectral clustering objective across $m$ views by cyclically solving single-view spectral clustering subproblems on adaptively modified Laplacians until the joint objective converges.
Input: Similarity matrices $K^{(1)}, \dots, K^{(m)} \in \mathbb{R}^{n \times n}$ , number of clusters $k$ , co-regularization trade-off parameter $\lambda > 0$ , convergence threshold $\epsilon = 10^{-4}$
Output: Cluster assignments for data points $1, \dots, n$
for $v = 1$ to $m$ do
    Compute diagonal degree matrix $D^{(v)}$ where $D_{ii}^{(v)} = \sum_{j=1}^n K_{ij}^{(v)}$
    Compute normalized Laplacian $\mathcal{L}^{(v)} = (D^{(v)})^{-1/2} K^{(v)} (D^{(v)})^{-1/2}$
    Initialize $U^{(v)} \in \mathbb{R}^{n \times k}$ as the top- $k$ eigenvectors of $\mathcal{L}^{(v)}$
end for
Compute initial objective value $\text{Obj}^{(0)} = \sum_{v=1}^m \text{tr}((U^{(v)})^T \mathcal{L}^{(v)} U^{(v)}) + \lambda \sum_{1 \le v \ne w \le m} \text{tr}(U^{(v)}(U^{(v)})^T U^{(w)}(U^{(w)})^T)$
$t \leftarrow 0$
repeat
     $t \leftarrow t + 1$
    for $v = 1$ to $m$ do
        Construct modified Laplacian $M^{(v)} = \mathcal{L}^{(v)} + \lambda \sum_{w=1, w \ne v}^m U^{(w)} (U^{(w)})^T$
        Compute $U^{(v)} \in \mathbb{R}^{n \times k}$ as the matrix of top- $k$ eigenvectors of $M^{(v)}$ such that $(U^{(v)})^T U^{(v)} = I_k$
    end for
     $\text{Obj}^{(t)} \leftarrow \sum_{v=1}^m \text{tr}((U^{(v)})^T \mathcal{L}^{(v)} U^{(v)}) + \lambda \sum_{1 \le v \ne w \le m} \text{tr}(U^{(v)}(U^{(v)})^T U^{(w)}(U^{(w)})^T)$
until $\text{Obj}^{(t)} - \text{Obj}^{(t-1)} < \epsilon$
Select embedding matrix $U^{(v)}$ from the most informative view (or concatenate $[U^{(1)}, \dots, U^{(m)}]$ )
Normalize each row of the selected embedding matrix to have unit Euclidean norm
Apply standard $k$ -means algorithm to the rows of the normalized embedding matrix
return cluster memberships from $k$ -means
Knowl 5 — Alternating Maximization Algorithm for Centroid-Based Co-regularized Spectral Clustering
algorithm
The centroid-based co-regularized multi-view spectral clustering algorithm alternates between updating view-specific spectral embedding matrices $U^{(v)}$ and updating a shared consensus centroid embedding matrix $U^*$ via standard eigensolvers.
Input: Similarity matrices $K^{(1)}, \dots, K^{(m)} \in \mathbb{R}^{n \times n}$ , view weights $\lambda_1, \dots, \lambda_m > 0$ , number of clusters $k$ , convergence threshold $\epsilon = 10^{-4}$
Output: Cluster assignments for data points $1, \dots, n$
for $v = 1$ to $m$ do
    Compute diagonal degree matrix $D^{(v)}$ where $D_{ii}^{(v)} = \sum_{j=1}^n K_{ij}^{(v)}$
    Compute normalized Laplacian $\mathcal{L}^{(v)} = (D^{(v)})^{-1/2} K^{(v)} (D^{(v)})^{-1/2}$
    Initialize $U^{(v)} \in \mathbb{R}^{n \times k}$ as the top- $k$ eigenvectors of $\mathcal{L}^{(v)}$
end for
Initialize consensus $U^* \in \mathbb{R}^{n \times k}$ as the top- $k$ eigenvectors of $\sum_{v=1}^m \lambda_v U^{(v)} (U^{(v)})^T$
Compute initial objective value $\text{Obj}^{(0)} = \sum_{v=1}^m \text{tr}((U^{(v)})^T \mathcal{L}^{(v)} U^{(v)}) + \sum_{v=1}^m \lambda_v \text{tr}(U^{(v)}(U^{(v)})^T U^*(U^*)^T)$
$t \leftarrow 0$
repeat
     $t \leftarrow t + 1$
    for $v = 1$ to $m$ do
        Construct modified Laplacian $M^{(v)} = \mathcal{L}^{(v)} + \lambda_v U^* (U^*)^T$
        Compute $U^{(v)} \in \mathbb{R}^{n \times k}$ as the matrix of top- $k$ eigenvectors of $M^{(v)}$ such that $(U^{(v)})^T U^{(v)} = I_k$
    end for
    Construct consensus Laplacian $M^* = \sum_{v=1}^m \lambda_v U^{(v)} (U^{(v)})^T$
    Compute consensus $U^* \in \mathbb{R}^{n \times k}$ as the top- $k$ eigenvectors of $M^*$ such that $(U^*)^T U^* = I_k$
     $\text{Obj}^{(t)} \leftarrow \sum_{v=1}^m \text{tr}((U^{(v)})^T \mathcal{L}^{(v)} U^{(v)}) + \sum_{v=1}^m \lambda_v \text{tr}(U^{(v)}(U^{(v)})^T U^*(U^*)^T)$
until $\text{Obj}^{(t)} - \text{Obj}^{(t-1)} < \epsilon$
Normalize each row of consensus matrix $U^*$ to unit Euclidean norm: $U^*_{ij} \leftarrow U^*_{ij} / \sqrt{\sum_l (U^*_{il})^2}$
Apply standard $k$ -means algorithm to the rows of the normalized consensus matrix $U^*$
return cluster memberships from $k$ -means
Knowl 6 — Convergence of Alternating Maximization in Co-regularized Spectral Clustering
theoretical result
For fixed sample size $n$ , number of clusters $k$ , and co-regularization trade-off parameters $\lambda$ (or view-specific weights $\lambda_v > 0$ ), the objective functions for both pairwise and centroid-based co-regularized spectral clustering are bounded from above by a finite constant because the Laplacians $\mathcal{L}^{(v)}$ have bounded spectra and all embedding matrices satisfy orthonormality constraints $(U^{(v)})^T U^{(v)} = I_k$ .

In each coordinate step of the alternating maximization algorithm, solving the top- $k$ eigenvector problem for the modified Laplacian yields the exact global maximum of the conditional subproblem for the targeted matrix variable while fixing all other variables. Therefore, the joint objective function is monotonically non-decreasing at each iteration $t$ :

$\text{Obj}^{(t)} \ge \text{Obj}^{(t-1)}$

Because the sequence of objective values is non-decreasing and bounded from above, the alternating maximization algorithm is guaranteed to converge to a stationary point (local maximum). In practice, convergence within a threshold of $\epsilon = 10^{-4}$ is reached in fewer than 10 iterations across evaluated datasets.

Knowl 7 — Multi-view Spectral Clustering Performance Across Synthetic and Real Datasets

data/table

The table below compares clustering quality measured by Normalized Mutual Information (NMI, ranging in $[0, 1]$ , higher is better) across two synthetic datasets and three real-world datasets: Reuters Multilingual (2 views: English and French), UCI Handwritten digits (2 views: Fourier coefficients and profile correlations), and Caltech-101 (up to 4 kernel views: pixel features, PHOG, sparse localized features, and SIFT). Numbers in parentheses are standard deviations over 20 random $k$ -means restarts. P denotes pairwise co-regularization and C denotes centroid-based co-regularization, with the number in parentheses indicating the number of views utilized.

Method	Synth data 1	Synth data 2	Reuters	Handwritten	Caltech
Best Single View	0.267 (0.0)	0.898 (0.0)	0.287 (0.019)	0.641 (0.008)	0.510 (0.008)
Feature Concat	0.294 (0.0)	0.923 (0.0)	0.298 (0.020)	0.619 (0.015)	–
Kernel Addition	0.339 (0.0)	0.973 (0.0)	0.323 (0.021)	0.744 (0.030)	0.383 (0.008)
Kernel Product	0.277 (0.0)	0.959 (0.0)	0.123 (0.010)	0.754 (0.026)	0.429 (0.007)
CCA	0.330 (0.0)	0.932 (0.0)	0.147 (0.003)	0.682 (0.019)	0.466 (0.007)
Min-Disagreement	0.313 (0.0)	0.936 (0.0)	0.342 (0.024)	0.745 (0.024)	0.389 (0.008)
Co-regularized (P) (2)	0.378 (0.0)	0.981 (0.0)	0.375 (0.002)	0.759 (0.031)	0.527 (0.007)
Co-regularized (P) (3)	–	0.989 (0.0)	–	–	0.533 (0.008)
Co-regularized (P) (4)	–	–	–	–	0.564 (0.007)
Co-regularized (C) (2)	0.367 (0.0)	0.955 (0.0)	0.360 (0.025)	0.768 (0.025)	0.522 (0.004)
Co-regularized (C) (3)	–	0.989 (0.0)	–	–	0.512 (0.007)
Co-regularized (C) (4)	–	–	–	–	0.561 (0.005)

Both proposed co-regularization approaches outperform single-view spectral clustering and multi-view baselines across all datasets. On Caltech-101, standard fusion baselines (Kernel Addition, CCA, Min-Disagreement) degrade below the single-view baseline (0.510), whereas co-regularized spectral clustering systematically improves performance as more views are incorporated (reaching 0.564 for pairwise and 0.561 for centroid-based with 4 views).

Knowl 8 — Sensitivity of Multi-view Clustering Performance to Co-regularization Parameter
empirical result
The clustering quality of pairwise co-regularized spectral clustering, evaluated by Normalized Mutual Information (NMI), is sensitive to the co-regularization trade-off parameter $\lambda$ but maintains superior performance over baseline methods across a broad interval $\lambda \in [0.01, 0.05]$ :
- On Reuters multilingual document clustering (English and French views), starting from $\lambda = 0$ (independent spectral clustering with NMI $\approx 0.287$ ), the NMI score increases sharply as $\lambda$ increases, peaking at $\lambda = 0.01$ (NMI $\approx 0.375$ ) with a secondary peak near $\lambda = 0.025$ . It then declines gradually and stays competitive with the best baseline (Minimizing-Disagreement, NMI $\approx 0.342$ ) until dropping below it only after $\lambda > 0.075$ .
- On Caltech-101 object image clustering, NMI increases rapidly from $\lambda = 0$ to reach its maximum near $\lambda = 0.01$ (NMI $\approx 0.564$ ), after which it gradually decreases with minor fluctuations while staying above the single-view baseline (0.510) and other baselines across the tested range $\lambda \le 0.10$ .
Centroid-based co-regularization exhibits analogous stability with respect to $\lambda_v$ .
Knowl 9 — Multi-view Benchmark Datasets and Preprocessing Setup
experimental setup
Evaluation of multi-view spectral clustering was conducted on two synthetic and three real-world datasets:
1. Synthetic Data 1 (2 views, 2 clusters): 1000 points sampled from two-component Gaussian mixture models per view. View 1 means: $\mu_1^{(1)} = (1, 1)$ , $\mu_2^{(1)} = (2, 2)$ ; covariances $\Sigma_1^{(1)} = \begin{pmatrix} 1 & 0.5 \\ 0.5 & 1.5 \end{pmatrix}$ , $\Sigma_2^{(1)} = \begin{pmatrix} 0.3 & 0 \\ 0 & 0.6 \end{pmatrix}$ . View 2 means: $\mu_1^{(2)} = (2, 2)$ , $\mu_2^{(2)} = (1, 1)$ ; covariances $\Sigma_1^{(2)} = \begin{pmatrix} 0.3 & 0 \\ 0 & 0.6 \end{pmatrix}$ , $\Sigma_2^{(2)} = \begin{pmatrix} 1 & 0.5 \\ 0.5 & 1.5 \end{pmatrix}$ .
2. Synthetic Data 2 (3 views, 2 clusters, correlated features): Two-component GMMs. Means: $\mu_1^{(1)} = (1, 1), \mu_2^{(1)} = (3, 4)$ ; $\mu_1^{(2)} = (1, 2), \mu_2^{(2)} = (2, 2)$ ; $\mu_1^{(3)} = (1, 1), \mu_2^{(3)} = (3, 3)$ . Covariances: $\Sigma_1^{(1)} = \begin{pmatrix} 1 & 0.5 \\ 0.5 & 1.5 \end{pmatrix}$ , $\Sigma_2^{(1)} = \begin{pmatrix} 0.3 & 0.2 \\ 0.2 & 0.6 \end{pmatrix}$ ; $\Sigma_1^{(2)} = \begin{pmatrix} 1 & -0.2 \\ -0.2 & 1 \end{pmatrix}$ , $\Sigma_2^{(2)} = \begin{pmatrix} 0.6 & 0.1 \\ 0.1 & 0.5 \end{pmatrix}$ ; $\Sigma_1^{(3)} = \begin{pmatrix} 1.2 & 0.2 \\ 0.2 & 1 \end{pmatrix}$ , $\Sigma_2^{(3)} = \begin{pmatrix} 1 & 0.4 \\ 0.4 & 0.7 \end{pmatrix}$ .
3. Reuters Multilingual Data: 1200 documents balanced across 6 categories (200 per class) from RCV1/RCV2. View 1 is original English, View 2 is French translation. Bag-of-words vectors are projected to 100 dimensions via Latent Semantic Analysis (LSA) prior to kernel computation.
4. UCI Handwritten Digits Data: 2000 images of digits 0--9. View 1 has 76 Fourier coefficients; View 2 has 216 profile correlations.
5. Caltech-101 Data: 450 images across 30 classes from the Multiple Kernel Learning repository using 4 kernel views: pixel features, Pyramid Histogram of Gradients (PHOG), Sparse Localized Features, and SIFT descriptors.
Similarities are computed with Gaussian kernels whose standard deviation $\sigma$ is set to the median pairwise Euclidean distance. Clustering quality is measured by Normalized Mutual Information (NMI) over 20 random initializations of $k$ -means.

Coverage note — No substantial contributed material was omitted from the paper.

References

1.A. Blum and T. Mitchell. Combining labeled and unlabeled data with co-training. In Conference on Learning Theory, 1998.
2.Kamalika Chaudhuri, Sham M. Kakade, Karen Livescu, and Karthik Sridharan. Multi-view Clustering via Canonical Correlation Analysis. In International Conference on Machine Learning, 2009.
3.Ulrike von Luxburg. A Tutorial on Spectral Clustering. Statistics and Computing, 2007.
4.J. Shi and J. Malik. Normalized cuts and Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22:888–905, 1997.
5.A. Ng, M. Jordan, and Y. Weiss. On spectral clustering: analysis and an algorithm. In Advances in Neural Information Processing Systems, 2002.
6.Vikas Sindhwani, Partha Niyogi, and Mikhail Belkin. A Co-regularization approach to semi-supervised learning with multiple views. In Proceedings of the Workshop on Learning with Multiple Views, International Conference on Machine Learning, 2005.
7.Alexander Strehl and Joydeep Ghosh. Cluster Ensembles - A Knowledge Reuse Framework for Combining Multiple Partitions. Journal of Machine Learning Research, pages 583–617, 2002.
8.Donglin Niu, Jennifer G. Dy, and Michael I. Jordan. Multiple non-redundant spectral clustering views. In International Conference on Machine Learning, 2010.
9.Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Learning non-linear combination of kernels. In Advances in Neural Information Processing Systems, 2009.
10.Matthew B. Blaschko and Christoph H. Lampert. Correlational Spectral Clustering. In Computer Vision and Pattern Recognition, 2008.
11.Virginia R. de Sa. Spectral Clustering with two views. In Proceedings of the Workshop on Learning with Multiple Views, International Conference on Machine Learning, 2005.
12.Xing Yi, Yunpeng Xu, and Changshui Zhang. Multi-view em algorithm for finite mixture models. In ICAPR, Lecture Notes in Computer Science, Springer-Verlag, 2005.
13.Massih-Reza Amini, Nicolas Usunier, and Cyril Goutte. Learning from multiple partially observed views - an application to multilingual text categorization. In Advances in Neural Information Processing Systems, 2009.
14.D. D. Lewis, Y. Yang, T. Rose, and F. Li. RCV1. A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361–397, 2004.
15.Reuters. Corpus, volume 2, multilingual corpus, 1996-08-20 to 1997-08-19, 2005.
16.Thomas Hofmann. Probabilistic latent semantic analysis. In Uncertainty in Artificial Intelligence, pages 289–296, 1999.
17.David M. Blei, Andreq Y. Ng, and Michael I. Jordan. Latent Dirichlet Allocation. Journal of Machine Learning Research, pages 993–1022, 2003.
18.The UCSD Multiple Kernel Learning Repository. http://mkl.ucsd.edu.
19.Steffen Bickel and Tobias Scheffer. Multi-View Clustering. In IEEE International Conference on Data Mining, 2004.
20.Dengyong Zhou and Christopher J. C. Burges. Spectral Clustering and Transductive Learning with Multiple Views. In International Conference on Machine Learning, 2007.
21.Abhishek Kumar and Hal Daume. A Co-training Approach for Multiview Spectral Clustering. In International Conference on Machine Learning, 2011.
22.Wei Tang, Zhengdong Lu, and Inderjit S. Dhillon. Clustering with Multiple Graphs. In IEEE International Conference on Data Mining, 2009.
23.Y. Bengio, P. Vincent, and J.F. Paiement. Spectral clustering and kernel PCA are learning eigenfunctions. Technical Report 2003s-19, CIRANO, 2003.
24.Ulrike von Luxburg, Mikhail Belkin, and Olivier Bousquet. Consistency of Spectral Clustering. Annals of Statistics, 36(2):555–586, 2008.

Citation

MLA

Kumar, A., et al. “Co-regularized Multi-view Spectral Clustering”. Advances in Neural Information Processing Systems, vol. 24, 2011, https://proceedings.neurips.cc/paper_files/paper/2011/file/31839b036f63806cba3f47b93af8ccb5-Paper.pdf.

APA

Kumar, A., Rai, P., & Daume, H. (2011). Co-regularized Multi-view Spectral Clustering. Advances in Neural Information Processing Systems, 24. https://proceedings.neurips.cc/paper_files/paper/2011/file/31839b036f63806cba3f47b93af8ccb5-Paper.pdf

Chicago

Kumar, A., P. Rai, and H. Daume. 2011. “Co-regularized Multi-view Spectral Clustering”. Advances in Neural Information Processing Systems 24. https://proceedings.neurips.cc/paper_files/paper/2011/file/31839b036f63806cba3f47b93af8ccb5-Paper.pdf.

Harvard

Kumar, A., Rai, P. and Daume, H. (2011) “Co-regularized Multi-view Spectral Clustering”, Advances in Neural Information Processing Systems. Curran Associates, Inc. Available at: https://proceedings.neurips.cc/paper_files/paper/2011/file/31839b036f63806cba3f47b93af8ccb5-Paper.pdf.

Vancouver

1. Kumar A, Rai P, Daume H (2011) Co-regularized Multi-view Spectral Clustering. Advances in Neural Information Processing Systems 24:

BibTeX

@inproceedings{kumar2011regularized,
  title = {Co-regularized Multi-view Spectral Clustering},
  author = {Kumar, Abhishek and Rai, Piyush and Daume, Hal},
  year = {2011},
  booktitle = {Advances in Neural Information Processing Systems},
  publisher = {Curran Associates, Inc.},
  volume = {24},
  url = {https://proceedings.neurips.cc/paper_files/paper/2011/file/31839b036f63806cba3f47b93af8ccb5-Paper.pdf}
}

Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

Co-regularized Multi-view Spectral Clustering

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Pairwise Co-regularized Multi-view Spectral Clustering

Knowl 2 — Centroid-Based Co-regularized Multi-view Spectral Clustering

Knowl 3 — Spectral Clustering Hypothesis Disagreement Metric

Knowl 4 — Alternating Maximization Algorithm for Pairwise Co-regularized Spectral Clustering

Knowl 5 — Alternating Maximization Algorithm for Centroid-Based Co-regularized Spectral Clustering

Knowl 6 — Convergence of Alternating Maximization in Co-regularized Spectral Clustering

Knowl 7 — Multi-view Spectral Clustering Performance Across Synthetic and Real Datasets

Knowl 8 — Sensitivity of Multi-view Clustering Performance to Co-regularization Parameter

Knowl 9 — Multi-view Benchmark Datasets and Preprocessing Setup

References

Citation

BibTeX

Access the Paper