Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

contrastive loss function

A contrastive loss function is a machine learning objective function used in distance metric learning to map data into an embedding space where similar items are placed close together and dissimilar items are pushed apart. Operating on pairs of input samples, it calculates a penalty based on their geometric distance in the feature space. For similar pairs, the function penalizes large distances to pull their representations together, whereas for dissimilar pairs, it penalizes representations that fall within a specified margin, enforcing separation. This formulation enables neural networks to learn invariant, low-dimensional representations based on relative relationships rather than explicit coordinate labels, allowing the model to accurately evaluate similarity for new and unseen inputs.

1 item

Dimensionality Reduction by Learning an Invariant Mapping

Dimensionality Reduction by Learning an Invariant Mapping

Raia Hadsell, Sumit Chopra, Yann LeCun

OrganizationsNew York University

Why you should read this

Proposes Dimensionality Reduction by Learning an Invariant Mapping (DrLIM), a contrastive learning framework that maps high-dimensional data into low-dimensional spaces while generalizing to unseen samples and remaining invariant to transformations without requiring predefined distance metrics.

Dimensionality reduction involves mapping a set of high dimensional input points onto a low dimensional manifold so that “similar” points in input space are mapped to nearby points on the manifold. Most existing techniques for solving the problem suffer from two drawbacks. First, most of them depend on a meaningful and computable distance metric in input space. Second, they do not compute a “function” that can accurately map new input samples whose relationship to the training data is unknown. We present a method - called Dimensionality Reduction by Learning an Invariant Mapping (DrLIM) - for learning a globally coherent non-linear function that maps the data evenly to the output manifold. The learning relies solely on neighborhood relationships and does not require any distance measure in the input space. The method can learn mappings that are invariant to certain transformations of the inputs, as is demonstrated with a number of experiments. Comparisons are made to other techniques, in particular LLE.

Added

2026-09-09