keyword
sparseness constraints
Sparseness constraints are mathematical conditions applied in optimization and machine learning algorithms that encourage or enforce representations, vectors, or matrix factors to contain a high proportion of zero or near-zero values. In techniques such as non-negative matrix factorization, dictionary learning, and dimensionality reduction, these constraints restrict the number of active components used to reconstruct or describe data. By limiting each observation to a combination of only a few meaningful elements, sparseness constraints promote distinct parts-based representations, enhance model interpretability, reduce complexity, and help prevent overfitting.
2 items
Orthogonal nonnegative matrix t-factorizations for clustering
C. Ding, Tao Li, Wei Peng, Haesun Park
Why you should read this
Establishes a rigorous mathematical foundation and convergent update algorithms for orthogonal three-factor nonnegative matrix factorization, enabling simultaneous, interpretable co-clustering of rows and columns in complex data matrices.
Currently, most research on nonnegative matrix factorization (NMF) focus on 2-factor X = FG^T factorization. We provide a systematic analysis of 3-factor X = FSG^T NMF. While unconstrained 3-factor NMF is equivalent to unconstrained 2-factor NMF, constrained 3-factor NMF brings new features to constrained 2-factor NMF. We study the orthogonality constraint because it leads to rigorous clustering interpretation. We provide new rules for updating F,S,G and prove the convergence of these algorithms. Experiments on 5 datasets and a real world case study are performed to show the capability of bi-orthogonal 3-factor NMF on simultaneously clustering rows and columns of the input data matrix. We provide a new approach of evaluating the quality of clustering on words using class aggregate distribution and multi-peak distribution. We also provide an overview of various NMF extensions and examine their relationships.
Added
2026-09-25
Non-negative Matrix Factorization with Sparseness Constraints
Patrik O. Hoyer
Why you should read this
Develops a non-negative matrix factorization framework with explicit sparseness constraints to reliably generate true parts-based linear representations of non-negative data.
Non-negative matrix factorization (NMF) is a recently developed technique for finding parts-based, linear representations of non-negative data. Although it has successfully been applied in several applications, it does not always result in parts-based representations. In this paper, we show how explicitly incorporating the notion of `sparseness' improves the found decompositions. Additionally, we provide complete MATLAB code both for standard NMF and for our extension. Our hope is that this will further the application of these methods to solving novel data-analysis problems.
Added
2026-09-13
