Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

hard negative examples

Hard negative examples are data samples that do not belong to the target class or match a given query, yet closely resemble positive instances in feature space, making them difficult for a machine learning model to distinguish. In contrast to easily separable negative samples that provide minimal feedback during training, hard negatives yield higher loss and stronger gradient updates. By exposing subtle differences near the decision boundary or semantic threshold, these challenging cases force the model to capture fine-grained characteristics and learn more robust, discriminative representations.

1 item

Unsupervised Sentence Representation via Contrastive Learning with Mixing Negatives

Unsupervised Sentence Representation via Contrastive Learning with Mixing Negatives

Yanzhao Zhang, Richong Zhang, Samuel Mensah, Xudong Liu, Yongyi Mao

OrganizationsAdvanced Innovation Center for Big Data and Brain ComputingBeihang UniversitySKLSDEUniversity of OttawaUniversity of Sheffield

Why you should read this

Proposes MixCSE, a contrastive sentence representation framework that overcomes vanishing gradient signals by continually generating artificial hard negative features through mixing positive and negative samples, achieving state-of-the-art results on semantic textual similarity and transfer tasks.

Unsupervised sentence representation learning is a fundamental problem in natural language processing. Recently, contrastive learning has made great success on this task. Existing contrastive learning based models usually apply random sampling to select negative examples for training. Previous work in computer vision has shown that hard negative examples help contrastive learning to achieve faster convergence and better optimization for representation learning. However, the importance of hard negatives in contrastive learning for sentence representation is yet to be explored. In this study, we prove that hard negatives are essential for maintaining strong gradient signals in the training process while random sampling negative examples is ineffective for sentence representation. Accordingly, we present a contrastive model, MixCSE, that extends the current state-of-the-art SimCSE by continually constructing hard negatives via mixing both positive and negative features. The superior performance of the proposed approach is demonstrated via empirical studies on Semantic Textual Similarity datasets and Transfer task datasets.

Added

2026-09-26