keyword
alignment loss
Alignment loss is an objective function in representation learning that measures the average distance between the feature embeddings of paired, semantically similar inputs. In self-supervised and contrastive learning frameworks where representations are mapped onto a unit hypersphere, alignment loss is calculated as the expected Euclidean distance between the normalized vectors of positive pairs, such as different augmented views of the same data sample. Minimizing this loss encourages the model to assign closely matching coordinates to related data points, enforcing invariance to task-irrelevant variations. To prevent representational collapse, where all inputs map to an identical point, alignment loss is typically coupled with uniformity objectives or negative sampling mechanisms that disperse unrelated embeddings evenly across the latent space.
1 item
