Back

Improved Deep Metric Learning with Multi-class N-pair Loss Objective

Kihyuk Sohn

article2016NeurIPS2,507 citations

Proposes the multi-class N-pair loss along with an efficient batch construction technique that jointly evaluates a query against multiple negative classes using only 2N forward passes, dramatically speeding up convergence and improving deep metric learning performance across retrieval, clustering, and verification tasks.

Listen

Deep metric learning seeks to embed data so that similar items are close and dissimilar items are far apart in a learned space. Existing methods based on contrastive or triplet loss often converge slowly because each update compares an example against only one negative class at a time. This limitation matters now because many visual recognition and verification tasks rely on deep embeddings yet still require slow training or extra data-mining steps.

The article set out to develop a loss function that jointly compares an input against multiple negative classes while remaining computationally practical for deep networks. The authors introduced an (N+1)-tuplet loss that generalizes the triplet objective and paired it with an efficient batch construction that needs only 2N examples to form N distinct tuplets. They called the resulting method multi-class N-pair loss and evaluated it against triplet loss with negative mining, other variants, and softmax classification.

Experiments covered fine-grained recognition on Car-333 and Flower-610, clustering and retrieval on Stanford Online Products, Car-196, and CUB-200, and face verification and identification on LFW using WebFace training data. Across these benchmarks the N-pair-mc loss produced higher accuracy than triplet baselines, often by substantial margins such as 9–14 points on recall and F1 metrics, while also converging in far fewer iterations. It matched or exceeded softmax recognition accuracy yet delivered markedly better verification performance.

These gains indicate that training embeddings with simultaneous comparison to many negatives yields more stable and discriminative representations without added mining cost on medium-scale problems. For very large numbers of classes the method still benefits from a proposed hard-negative class selection step. The results matter because faster, stronger metric learning directly improves downstream tasks such as image search, clustering, and identity verification.

Practitioners should adopt the N-pair-mc formulation as the default objective for new deep metric learning work and reserve negative class mining for the largest label sets. Further gains are likely from combining the loss with larger training sets or modern network backbones. The main limitations are that all reported results use GoogLeNet or CasiaNet architectures on fixed datasets; broader validation across additional domains and scales would increase confidence.

NEC Laboratories America, Inc.association rulesclassificationclustering algorithmsmulti-class N-pair lossdeep metric learningtriplet loss models
Sohn (2016).pdf
Cover for Improved Deep Metric Learning with Multi-class N-pair Loss Objective

Abstract

Deep metric learning has gained much popularity in recent years, following the success of deep learning. However, existing frameworks of deep metric learning based on contrastive loss and triplet loss often suffer from slow convergence, partially because they employ only one negative example while not interacting with the other negative classes in each update. In this paper, we propose to address this problem with a new metric learning objective called multi-class N-pair loss. The proposed objective function firstly generalizes triplet loss by allowing joint comparison among more than one negative examples – more specifically, N-1 negative examples – and secondly reduces the computational burden of evaluating deep embedding vectors via an efficient batch construction strategy using only N pairs of examples, instead of (N+1)×N. We demonstrate the superiority of our proposed loss to the triplet loss as well as other competing loss functions for a variety of tasks on several visual recognition benchmark, including fine-grained object recognition and verification, image clustering and retrieval, and face verification and identification.

Topics

association rulesclassificationclustering algorithmsgenetic selectionidentity verificationimage searchonline productstraining samplevalidationverification method

Keywords

multi-class N-pair lossdeep metric learningtriplet loss modelscontrastive lossnegative class miningefficient batch constructionunconstrained face recognitionCUB-200 datasets

Table of Contents

  • 1 Introduction
  • 2 Preliminary: Distance Metric Learning
  • 3 Deep Metric Learning with Multiple Negative Examples
  • 3.1 Learning to identify from multiple negative examples
  • 3.2 NN-pair loss for efficient deep metric learning
  • 3.2.1 Hard negative class mining
  • 3.2.2 L² norm regularization of embedding vectors
  • 4 Experimental Results
  • 4.1 Fine-grained visual object recognition and verification
  • 4.2 Distance metric learning for unseen object recognition
  • 4.3 Face verification and identification
  • 4.4 Analysis on tuplet construction methods
  • 5 Conclusion
  • Acknowledgments
  • References

Knowls

  1. Knowl 1 — Multi-Class N-Pair Loss Formulation

    equation

    Given NN pairs of training examples from NN mutually distinct classes, denoted as {(x1,x1+),…,(xN,xN+)}\{(x_1, x_1^+), \dots, (x_N, x_N^+)\} where labels satisfy yi≠yjy_i \neq y_j for all i≠ji \neq j, and an embedding function f(⋅;θ)∈RKf(\cdot; \theta) \in \mathbb{R}^K parameterized by deep neural network weights θ\theta, the multi-class NN-pair loss (denoted as LN-pair-mc\mathcal{L}_{\text{N-pair-mc}}) is defined as:

    LN-pair-mc({(xi,xi+)}i=1N;f)=1N∑i=1Nlog⁡(1+∑j≠iexp⁡(fi⊤fj+−fi⊤fi+))\mathcal{L}_{\text{N-pair-mc}}(\{(x_i, x_i^+)\}_{i=1}^N; f) = \frac{1}{N} \sum_{i=1}^N \log \left( 1 + \sum_{j \neq i} \exp(f_i^\top f_j^+ - f_i^\top f_i^+) \right)

    where fi=f(xi;θ)f_i = f(x_i; \theta) represents the query embedding for class ii, fi+=f(xi+;θ)f_i^+ = f(x_i^+; \theta) represents the positive embedding for class ii, and fj+=f(xj+;θ)f_j^+ = f(x_j^+; \theta) (j≠ij \neq i) represent the negative embeddings originating from the N−1N-1 other classes in the batch.

    The objective can also include its symmetric counterpart that swaps query representations fif_i and positive representations fi+f_i^+ to maximize training efficacy.

  2. Knowl 2 — N-Pair Mini-Batch Construction for Scalable Multi-Class Tuplet Loss

    model/method

    Evaluating (N+1)(N+1)-tuplet loss naively on a mini-batch of size MM requires passing M×(N+1)M \times (N+1) examples through deep network ff, which grows quadratically with NN and becomes intractable for large batch sizes.

    The NN-pair batch construction resolves this by sampling exactly NN pairs of training examples {(x1,x1+),…,(xN,xN+)}\{(x_1, x_1^+), \dots, (x_N, x_N^+)\} from NN mutually disjoint classes (yi≠yjy_i \neq y_j for all i≠ji \neq j). From these 2N2N sampled instances, NN distinct (N+1)(N+1)-tuplets {Si}i=1N\{S_i\}_{i=1}^N are formed:

    Si={xi,xi+,x1+,…,xi−1+,xi+1+,…,xN+}S_i = \{x_i, x_i^+, x_1^+, \dots, x_{i-1}^+, x_{i+1}^+, \dots, x_N^+\}

    In each tuplet SiS_i, xix_i acts as the anchor query, xi+x_i^+ acts as the positive instance, and the positive instances {xj+}j≠i\{x_j^+\}_{j \neq i} of all other classes act as N−1N-1 negative instances. Consequently, evaluating NN distinct (N+1)(N+1)-tuplets requires only 2N2N forward passes through the embedding network rather than (N+1)N(N+1)N passes.

  3. Knowl 3 — (N+1)-Tuplet Loss Objective

    equation

    For a single query instance xx, its positive instance x+x^+, and N−1N-1 negative instances {x1,…,xN−1}\{x_1, \dots, x_{N-1}\} sampled from N−1N-1 distinct negative classes, the (N+1)(N+1)-tuplet loss is defined as:

    L({x,x+,{xi}i=1N−1};f)=log⁡(1+∑i=1N−1exp⁡(f⊤fi−f⊤f+))\mathcal{L}(\{x, x^+, \{x_i\}_{i=1}^{N-1}\}; f) = \log \left( 1 + \sum_{i=1}^{N-1} \exp(f^\top f_i - f^\top f^+) \right)

    where f=f(x;θ)∈RKf = f(x; \theta) \in \mathbb{R}^K, f+=f(x+;θ)∈RKf^+ = f(x^+; \theta) \in \mathbb{R}^K, and fi=f(xi;θ)∈RKf_i = f(x_i; \theta) \in \mathbb{R}^K are embedding vectors produced by a deep neural network parameterized by θ\theta. This objective pushes all N−1N-1 negative examples away simultaneously while pulling the positive example closer, with the magnitude of each push proportional to the negative example's similarity to the query.

  4. Knowl 4 — Relation of (N+1)-Tuplet Loss to Triplet Loss and Multi-Class Logistic Partition Function

    theoretical result

    The (N+1)(N+1)-tuplet loss connects directly to both triplet loss and multi-class logistic (softmax) loss under the following theoretical relationships:

    1. Equivalence to Triplet Loss at N=2N=2: When N=2N=2, the (2+1)(2+1)-tuplet loss is given by:

    L(2+1)-tuplet({x,x+,xi};f)=log⁡(1+exp⁡(f⊤fi−f⊤f+))\mathcal{L}_{(2+1)\text{-tuplet}}(\{x, x^+, x_i\}; f) = \log \left(1 + \exp(f^\top f_i - f^\top f^+)\right)

    Assuming unit-norm embedding vectors (∣∣f∣∣2=1||f||_2 = 1), an embedding kernel ff minimizes L(2+1)-tuplet\mathcal{L}_{(2+1)\text{-tuplet}} if and only if it minimizes the smooth upper bound of the standard triplet loss Ltriplet({x,x+,xi};f)=max⁡(0,f⊤fi−f⊤f+)\mathcal{L}_{\text{triplet}}(\{x, x^+, x_i\}; f) = \max(0, f^\top f_i - f^\top f^+).

    1. Partition Function Approximation for N>2N > 2: For an ideal (L+1)(L+1)-tuplet loss defined over all L−1L-1 negative classes in a dataset:

    log⁡(1+∑i=1L−1exp⁡(f⊤fi−f⊤f+))=−log⁡exp⁡(f⊤f+)exp⁡(f⊤f+)+∑i=1L−1exp⁡(f⊤fi)\log \left( 1 + \sum_{i=1}^{L-1} \exp(f^\top f_i - f^\top f^+) \right) = -\log \frac{\exp(f^\top f^+)}{\exp(f^\top f^+) + \sum_{i=1}^{L-1} \exp(f^\top f_i)}

    This formulation is equivalent to the multi-class logistic loss with true partition function Z=exp⁡(f⊤f+)+∑i=1L−1exp⁡(f⊤fi)Z = \exp(f^\top f^+) + \sum_{i=1}^{L-1} \exp(f^\top f_i). The (N+1)(N+1)-tuplet loss uses N−1N-1 sampled negative classes to estimate this partition function. As NN increases toward LL, the sample approximation becomes monotonically more accurate, making (N+1)(N+1)-tuplet loss a tighter approximation to the global multi-class logistic loss than standard triplet loss (N=2N=2).

  5. Knowl 5 — Hard Negative Class Mining Algorithm for N-Pair Metric Learning

    algorithm

    When training on datasets with a very large number of classes LL, instance-level mining for NN mutually negative classes is computationally intractable. Hard negative class mining greedily selects NN classes that maximally violate distance metric constraints.

    Input: Total number of classes LL, target batch class count NN, candidate pool size CC (C≫NC \gg N), embedding model f(⋅;θ)f(\cdot; \theta)
    Output: Mini-batch of NN pairs {(xi,xi+)}i=1N\{(x_i, x_i^+)\}_{i=1}^N
    Randomly select a subset of CC classes from the total LL classes
    for each class c∈{1,…,C}c \in \{1, \dots, C\} do
        Sample 1 or 2 instances and compute their embedding vectors using ff
    end for
    Randomly select the initial class c1c_1 from candidate pool CC
    Initialize selected class set S←{c1}\mathcal{S} \leftarrow \{c_1\}
    while ∣S∣<N|\mathcal{S}| < N do
        Find candidate class c∗∈C∖Sc^* \in C \setminus \mathcal{S} that maximally violates the triplet constraint with respect to the currently selected classes in S\mathcal{S}
        if multiple candidate classes tie for maximum violation then
            Select one tied candidate class uniformly at random
        end if
        S←S∪{c∗}\mathcal{S} \leftarrow \mathcal{S} \cup \{c^*\}
    end while
    for each class c∈Sc \in \mathcal{S} do
        Draw two instances (xc,xc+)(x_c, x_c^+) from class cc
    end for
    return {(xc,xc+)}c∈S\{(x_c, x_c^+)\}_{c \in \mathcal{S}}

  6. Knowl 6 — One-vs-One N-Pair Loss Formulation

    equation

    When the NN-pair batch construction of NN pairs {(xi,xi+)}i=1N\{(x_i, x_i^+)\}_{i=1}^N from NN different classes is combined with independent pairwise (triplet-style) comparisons rather than joint multi-class identification, the resulting one-vs-one NN-pair loss (denoted as LN-pair-ovo\mathcal{L}_{\text{N-pair-ovo}}) is formulated as:

    LN-pair-ovo({(xi,xi+)}i=1N;f)=1N∑i=1N∑j≠ilog⁡(1+exp⁡(fi⊤fj+−fi⊤fi+))\mathcal{L}_{\text{N-pair-ovo}}(\{(x_i, x_i^+)\}_{i=1}^N; f) = \frac{1}{N} \sum_{i=1}^N \sum_{j \neq i} \log \left( 1 + \exp(f_i^\top f_j^+ - f_i^\top f_i^+) \right)

    where fi=f(xi;θ)f_i = f(x_i; \theta) and fi+=f(xi+;θ)f_i^+ = f(x_i^+; \theta) are embedding vectors generated by deep network ff. Unlike the multi-class formulation LN-pair-mc\mathcal{L}_{\text{N-pair-mc}}, LN-pair-ovo\mathcal{L}_{\text{N-pair-ovo}} decouples negative classes and accumulates separate losses for each negative example independently, omitting the joint competition among negative classes.

  7. Knowl 7 — Embedding L2 Norm Regularization for Dot-Product Distance Metric Learning

    model/method

    In dot-product similarity metrics f⊤f+f^\top f^+, the similarity depends on both the angle and the magnitudes of the embedding vectors. While projecting embeddings onto the unit hypersphere (L2L_2 normalization) eliminates magnitude variance, it restricts ∣f⊤f+∣≤1|f^\top f^+| \le 1, which restricts gradient dynamics and hampers optimization during training of multi-class (N+1)(N+1)-tuplet objectives.

    To prevent vectors from growing arbitrarily while preserving unconstrained optimization dynamics, the embedding vectors are kept unnormalized, and an explicit L2L_2 norm regularization penalty is added to regularize the L2L_2 norm of the embedding vectors f(x;θ)f(x; \theta) to remain small.

  8. Knowl 8 — Clustering and Retrieval Performance on Unseen Object Benchmarks

    data/table

    Models were evaluated on zero-shot clustering (F1 score and Normalized Mutual Information, NMI) and image retrieval (Recall@K) where training and test object categories are disjoint. Networks were initialized with ImageNet-pretrained GoogLeNet, producing 64-dimensional embeddings for Car-196 (first 98 classes for training, remaining 98 for testing) and CUB-200 (first 100 classes for training, remaining 100 for testing), and 512-dimensional embeddings for Stanford Online Products (11,318 classes for training, 11,316 classes for testing).

    Online Products triplet triplet-nm triplet-lifted [21] 60-pair-ovo 60-pair-ovo-nm 60-pair-mc 60-pair-mc-nm
    F1 19.59 24.27 25.60 23.13 25.31 26.53 28.19
    NMI 86.11 87.23 87.50 86.98 87.45 87.77 88.10
    Recall@1 53.32 62.39 61.80 60.71 63.85 65.25 67.73
    Recall@10 72.75 79.69 79.90 78.74 81.22 82.15 83.76
    Recall@100 87.66 91.10 91.10 91.03 91.89 92.60 92.98
    Recall@1000 96.43 97.25 97.30 97.50 97.51 97.92 97.81
    Car-196 triplet triplet-nm – 60-pair-ovo – 60-pair-mc –
    F1 24.73 27.86 – 33.52 – 33.55 –
    NMI 58.25 59.94 – 63.87 – 63.95 –
    Recall@1 53.84 61.62 – 69.52 – 71.12 –
    Recall@2 66.02 73.48 – 78.76 – 79.74 –
    Recall@4 75.91 81.88 – 85.80 – 86.48 –
    Recall@8 84.18 87.81 – 90.94 – 91.60 –
    CUB-200 triplet triplet-nm – 60-pair-ovo – 60-pair-mc –
    F1 21.88 24.37 – 25.21 – 27.24 –
    NMI 55.83 57.87 – 58.55 – 60.39 –
    Recall@1 43.30 46.47 – 48.73 – 50.96 –
    Recall@2 55.84 58.58 – 60.48 – 63.34 –
    Recall@4 67.30 71.03 – 72.08 – 74.29 –
    Recall@8 77.48 80.17 – 81.62 – 83.22 –

    The 60-pair-mc model consistently outperforms standard triplet loss, hard negative mined triplet loss (triplet-nm), and lifted structured feature embedding across all datasets. On Online Products, 60-pair-mc with hard negative class mining (-nm) improves F1 by 8.60%, NMI by 1.99%, and Recall@1 by 14.41% over vanilla triplet loss.

  9. Knowl 9 — Recognition and Verification Performance on Car-333 and Flower-610

    data/table

    Metric learning objectives and softmax loss were compared on Car-333 (157,023 training / 7,840 test images across 333 categories) and Flower-610 (58,721 training / 3,050 test images across 610 categories). Models were trained with GoogLeNet for 40,000 iterations using a mini-batch size of 144 (72 pairs). Recognition was measured using a kkNN classifier (and a softmax classifier where indicated by †\dagger). Verification accuracy (VRF) measures whether a query is closer to its true positive counterpart than to Nneg∈{1,71}N_{\text{neg}} \in \{1, 71\} negative examples from different classes.

    Dataset Metric triplet triplet-nm 72-pair-ovo 72-pair-mc softmax
    Car-333 Recognition 70.24±0.3870.24 \pm 0.38 83.22±0.0983.22 \pm 0.09 86.84±0.1386.84 \pm 0.13 88.37±0.0588.37 \pm 0.05 89.21±0.1689.21 \pm 0.16 (88.69±0.20†88.69 \pm 0.20^\dagger)
    VRF (Nneg=1N_{\text{neg}}=1) 96.78±0.0496.78 \pm 0.04 97.39±0.0797.39 \pm 0.07 98.09±0.0798.09 \pm 0.07 97.92±0.0697.92 \pm 0.06 96.19±0.0796.19 \pm 0.07
    VRF (Nneg=71N_{\text{neg}}=71) 48.96±0.3548.96 \pm 0.35 65.14±0.2465.14 \pm 0.24 73.05±0.2573.05 \pm 0.25 76.02±0.30\mathbf{76.02 \pm 0.30} 55.36±0.3055.36 \pm 0.30
    Flower-610 Recognition 71.55±0.2671.55 \pm 0.26 82.85±0.2282.85 \pm 0.22 84.10±0.4284.10 \pm 0.42 85.57±0.25\mathbf{85.57 \pm 0.25} 84.38±0.2884.38 \pm 0.28 (84.59±0.21†84.59 \pm 0.21^\dagger)
    VRF (Nneg=1N_{\text{neg}}=1) 98.73±0.0398.73 \pm 0.03 99.15±0.0399.15 \pm 0.03 99.32±0.0399.32 \pm 0.03 99.50±0.02\mathbf{99.50 \pm 0.02} 98.72±0.0498.72 \pm 0.04
    VRF (Nneg=71N_{\text{neg}}=71) 73.04±0.1373.04 \pm 0.13 83.13±0.1583.13 \pm 0.15 87.42±0.1887.42 \pm 0.18 88.63±0.14\mathbf{88.63 \pm 0.14} 78.44±0.3378.44 \pm 0.33

    The 72-pair-mc model achieves recognition performance competitive with softmax classification while substantially outperforming softmax loss on multi-negative verification (Nneg=71N_{\text{neg}}=71), improving VRF by +20.66%+20.66\% on Car-333 and +10.19%+10.19\% on Flower-610.

  10. Knowl 10 — Face Verification, Closed-Set, and Open-Set Identification on LFW

    data/table

    Models were trained on CASIA-WebFace (494,414 images across 10,575 identities) using the CasiaNet architecture trained from scratch for 240,000 iterations and evaluated on Labeled Faces in the Wild (LFW). Evaluated metrics include verification accuracy (VRF), Rank-1 closed-set identification accuracy, and open-set identification detection and identification rate at a false accept rate of 1% (DIR@FAR=1%). The batch size was 384 images for all models except 320-pair-mc (640 images).

    Metric triplet triplet-nm 192-pair-ovo 192-pair-mc 320-pair-mc
    VRF (%) 95.88±0.3095.88 \pm 0.30 96.68±0.3096.68 \pm 0.30 96.92±0.2496.92 \pm 0.24 98.27±0.1998.27 \pm 0.19 98.33±0.17\mathbf{98.33 \pm 0.17}
    Rank-1 (%) 55.14 60.93 66.21 88.58 90.17
    DIR@FAR=1% (%) 25.96 34.60 34.14 66.51 71.76

    The 192-pair-mc loss substantially outperforms vanilla triplet and negative-mined triplet baselines, improving Rank-1 closed-set identification from 55.14%55.14\% to 88.58%88.58\% and DIR@FAR=1% from 25.96%25.96\% to 66.51%66.51\%. Scaling to 320-pair-mc further improves performance to 98.33%98.33\% VRF, 90.17%90.17\% Rank-1, and 71.76%71.76\% DIR@FAR=1%, surpassing the CasiaNet baseline trained with combined contrastive and softmax loss (96.13%96.13\% VRF).

  11. Knowl 11 — Impact of Class Diversity vs. Positive Instances per Class in Mini-Batches

    data/table

    To evaluate the trade-off between the number of distinct classes NN and the number of positive instances per class MM under a fixed mini-batch budget, batches were constructed with configurations N×MN \times M (where N×MN \times M is held constant). To accommodate M>2M > 2, the Neighborhood Component Analysis (NCA) formulation was used:

    L=1N⋅M∑i−log⁡∑j≠i:yj=yiexp⁡(fi⊤fj)∑j≠iexp⁡(fi⊤fj)\mathcal{L} = \frac{1}{N \cdot M} \sum_{i} -\log \frac{\sum_{j \neq i: y_j = y_i} \exp(f_i^\top f_j)}{\sum_{j \neq i} \exp(f_i^\top f_j)}

    Online Products Car-196 CUB-200
    Config (N×MN \times M) 60×260 \times 2 30×430 \times 4 60×260 \times 2 30×430 \times 4 10×1210 \times 12 60×260 \times 2 30×430 \times 4 10×1210 \times 12
    F1 26.53 25.01 33.55 31.92 29.87 27.24 27.54 26.66
    NMI 87.77 87.40 63.87 62.94 61.84 60.39 60.43 59.37
    Recall@1 65.25 63.58 71.12 69.30 65.49 50.96 50.91 49.65
    LFW Evaluation (N×MN \times M) 192×2192 \times 2 96×496 \times 4 64×664 \times 6 32×1232 \times 12
    VRF (%) 98.27±0.19\mathbf{98.27 \pm 0.19} 98.25±0.2598.25 \pm 0.25 97.98±0.2297.98 \pm 0.22 97.57±0.3397.57 \pm 0.33
    Rank-1 (%) 88.58 87.53 83.96 79.61
    DIR@FAR=1% (%) 66.51 66.22 64.38 56.46

    Across all benchmarks, decreasing the number of distinct classes NN while increasing instances per class MM degrades metric learning performance, establishing that maximizing negative class diversity in each mini-batch is more advantageous than sampling additional positive instances per class.

  12. Knowl 12 — Convergence Acceleration of Multi-Class N-Pair Loss over Triplet Loss

    empirical result

    When training CasiaNet on the CASIA-WebFace database, the 192-pair multi-class loss (192-pair-mc) achieves substantially faster convergence than the triplet loss model. The 192-pair-mc model reaches the loss level that the standard triplet loss achieves at convergence (after 240,000 iterations) within only 15,000 iterations, corresponding to a 16-fold reduction in training iterations.

Coverage note — No substantial contributed material was omitted. All primary losses, batch construction techniques, mining algorithms, theoretical connections, and experimental benchmarks across fine-grained recognition, retrieval, face verification, and ablations are covered.

References

  1. 1.L. Best-Rowden, H. Han, C. Otto, B. F. Klare, and A. K. Jain. Unconstrained face recognition: Identifying a person of interest from a media collection. IEEE Transactions on Information Forensics and Security, 9(12):2144–2157, 2014.
  2. 2.G. Chechik, V. Sharma, U. Shalit, and S. Bengio. Large scale online learning of image similarity through ranking. Journal of Machine Learning Research, 11:1109–1135, 2010.
  3. 3.S. Chopra, R. Hadsell, and Y. LeCun. Learning a similarity metric discriminatively, with application to face verification. In CVPR, 2005.
  4. 4.Y. Cui, F. Zhou, Y. Lin, and S. Belongie. Fine-grained categorization and dataset bootstrapping using deep metric learning with humans in the loop. In CVPR, 2016.
  5. 5.R. Girshick, J. Donahue, T. Darrell, and J. Malik. Region-based convolutional networks for accurate object detection and segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, PP(99):1–1, 2015.
  6. 6.J. Goldberger, G. E. Hinton, S. T. Roweis, and R. Salakhutdinov. Neighbourhood components analysis. In NIPS, 2004.
  7. 7.R. Hadsell, S. Chopra, and Y. LeCun. Dimensionality reduction by learning an invariant mapping. In CVPR, 2006.
  8. 8.G. B. Huang, M. Narayana, and E. Learned-Miller. Towards unconstrained face recognition. In CVPR Workshop, 2008.
  9. 9.H. Jegou, M. Douze, and C. Schmid. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):117–128, 2011.
  10. 10.Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.
  11. 11.D. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
  12. 12.J. Krause, M. Stark, J. Deng, and L. Fei-Fei. 3d object representations for fine-grained categorization. In ICCV Workshop, 2013.
  13. 13.A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, 2012.
  14. 14.J. Liu, Y. Deng, T. Bai, and C. Huang. Targeting ultimate accuracy: Face recognition via deep embedding. CoRR, abs/1506.07310, 2015.
  15. 15.D. G. Lowe. Similarity metric learning for a variable-kernel classifier. Neural computation, 7(1):72–85, 1995.
  16. 16.C. D. Manning, P. Raghavan, H. Schütze, et al. Introduction to information retrieval, volume 1. Cambridge university press Cambridge, 2008.
  17. 17.M. Norouzi, D. J. Fleet, and R. R. Salakhutdinov. Hamming distance metric learning. In NIPS, 2012.
  18. 18.O. M. Parkhi, A. Vedaldi, and A. Zisserman. Deep face recognition. BMVC, 2015.
  19. 19.F. Schroff, D. Kalenichenko, and J. Philbin. FaceNet: A unified embedding for face recognition and clustering. In CVPR, 2015.
  20. 20.K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
  21. 21.H. O. Song, Y. Xiang, S. Jegelka, and S. Savarese. Deep metric learning via lifted structured feature embedding. In CVPR, 2016.
  22. 22.Y. Sun, Y. Chen, X. Wang, and X. Tang. Deep learning face representation by joint identification-verification. In NIPS, 2014.
  23. 23.C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, 2015.
  24. 24.Y. Taigman, M. Yang, M. Ranzato, and L. Wolf. Deepface: Closing the gap to human-level performance in face verification. In CVPR, 2014.
  25. 25.C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011.
  26. 26.J. Wang, Y. Song, T. Leung, C. Rosenberg, J. Wang, J. Philbin, B. Chen, and Y. Wu. Learning fine-grained image similarity with deep ranking. In CVPR, 2014.
  27. 27.K. Q. Weinberger, J. Blitzer, and L. K. Saul. Distance metric learning for large margin nearest neighbor classification. In NIPS, 2005.
  28. 28.J. Weston, S. Bengio, and N. Usunier. Wsabie: Scaling up to large vocabulary image annotation. In IJCAI, volume 11, pages 2764–2770, 2011.
  29. 29.S. Xie, T. Yang, X. Wang, and Y. Lin. Hyper-class augmented and regularized deep learning for fine-grained image classification. In CVPR, 2015.
  30. 30.E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell. Distance metric learning with application to clustering with side-information. 2003.
  31. 31.D. Yi, Z. Lei, S. Liao, and S. Z. Li. Learning face representation from scratch. CoRR, abs/1411.7923, 2014.
  32. 32.X. Zhang, F. Zhou, Y. Lin, and S. Zhang. Embedding label structures for fine-grained feature representation. In CVPR, 2016.

Citation

MLA
Sohn, K. “Improved Deep Metric Learning with Multi-class N-pair Loss Objective”. Neural Information Processing Systems, vol. 29, 2016, pp. 1857–65, https://papers.nips.cc/paper/2016/file/6b180037abbebea991d8b1232f8a8ca9-Paper.pdf.
APA
Sohn, K. (2016). Improved deep metric learning with multi-class N-pair loss objective. Neural Information Processing Systems, 29, 1857–1865. https://papers.nips.cc/paper/2016/file/6b180037abbebea991d8b1232f8a8ca9-Paper.pdf
Chicago
Sohn, K. 2016. “Improved Deep Metric Learning with Multi-class N-pair Loss Objective”. Neural Information Processing Systems 29: 1857–65. https://papers.nips.cc/paper/2016/file/6b180037abbebea991d8b1232f8a8ca9-Paper.pdf.
Harvard
Sohn, K. (2016) “Improved deep metric learning with multi-class N-pair loss objective”, Neural Information Processing Systems, 29, pp. 1857–1865. Available at: https://papers.nips.cc/paper/2016/file/6b180037abbebea991d8b1232f8a8ca9-Paper.pdf.
Vancouver
1. Sohn K (2016) Improved deep metric learning with multi-class N-pair loss objective. Neural Information Processing Systems 29:1857–1865

BibTeX

@article{sohn2016improved,
  title = {Improved deep metric learning with multi-class N-pair loss objective},
  author = {Sohn, Kihyuk},
  year = {2016},
  journal = {Neural Information Processing Systems},
  volume = {29},
  pages = {1857-1865},
  url = {https://papers.nips.cc/paper/2016/file/6b180037abbebea991d8b1232f8a8ca9-Paper.pdf}
}
Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors