keyword
mixed negatives
Mixed negatives are synthetically generated negative examples created in contrastive representation learning by interpolating or combining the feature representations of existing data points, such as blending features between positive and negative instances or across multiple negative instances. In contrastive learning, a model learns meaningful representations by pulling similar examples together in an embedding space while pushing dissimilar negative examples apart. Standard negative samples selected at random often provide weak learning signals because they are trivially easy for the network to differentiate. By synthesizing mixed negative embeddings directly in the latent space, practitioners introduce harder negative examples that lie closer to the target representations. This approach maintains informative gradient signals throughout training and encourages models to capture more robust and fine-grained distinctions across tasks in natural language processing and computer vision without requiring excessively large memory banks or additional annotated data.
1 item
