Back

MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition

Yandong GuoLei ZhangYuxiao HuXiaodong HeJianfeng Gao

article2016ECCV2,231 citations

Introduces MS-Celeb-1M, a dataset of ten million images spanning one million celebrities linked to a knowledge base, providing the scale and evaluation protocol needed to train and benchmark real-world face recognition models.

Listen

Facial recognition research faces two major bottlenecks: public academic datasets are significantly smaller than proprietary industrial collections, and conventional systems focus primarily on visual similarity rather than disambiguating identities. Standard models often identify matching features without answering specifically who is pictured, confusing distinct individuals who share names or similar appearances. The article introduces MS-Celeb-1M, a large-scale benchmark and public dataset designed to recognize one million public figures from facial images by directly linking each detected face to a unique entity identifier in an established knowledge base.

To construct this resource, the authors extracted one million real-world public figures from the Freebase knowledge base, ranked by their web occurrence frequency to reflect practical interest across diverse professions, nationalities, and demographics. For model training, they assembled a primary dataset of approximately 10 million web-retrieved images covering the top 100,000 celebrities, providing both aligned and unaligned cropped faces. To evaluate performance, they created a rigorously vetted measurement set combining randomly selected test images, hard generalization cases, and distractor images to obscure target identities and test recognition coverage at a fixed high precision threshold of 95%.

The initial findings demonstrate both the viability and the steep challenge of web-scale identity linking. Using a deep neural network baseline trained on the 100,000-entity dataset, the model achieved a 72.8% coverage rate on randomly selected evaluation images at 95% precision. However, when tested against the more demanding hard evaluation set, coverage fell sharply to 44.2% at 95% precision, and dropped to just 5.2% at an ultra-high precision threshold of 99%. Because the initial training release covers only the top 10% most frequent entities of the million-person target list, the baseline system faces an inherent recall ceiling of roughly 75% on the overall measurement set.

These results confirm that while massive, automatically retrieved web data enables deep learning models to achieve strong initial accuracy, substantial performance degradation occurs on non-standard facial angles, aging, and less frequently occurring individuals. For decision-makers and technology leaders, closing the gap between general face matching and exact entity linking reduces operational and reputational risk across applications such as media tagging, automated search indexing, and content understanding. Linking faces directly to rich knowledge-graph metadata provides structured context that pure visual matching cannot supply.

The article recommends that organizations and researchers expand data collection to cover the long tail of less prominent individuals and develop robust techniques for cleaning noisy web labels and managing large-scale classification outputs. Organizations planning to deploy web-scale recognition systems should anticipate the need for outside data integration or dedicated model fine-tuning to surpass the 75% coverage ceiling. While high confidence exists in the baseline methodology and evaluation rigor, stakeholders should exercise caution regarding performance on unconstrained or rare individuals until automated data-cleaning and model scalability are further improved.

Microsoftclassificationcomputer visionfacial imagesFaceImagesRecognition
arXiv: 1607.08221
  • Paper: FaceNet: A unified embedding for face recognition and clustering, Florian Schroff et al. (2015). FaceNet establishes the foundational deep metric learning paradigm and triplet loss methodology that later massive face recognition datasets like MS-Celeb-1M are designed to train.
  • Paper: Deep Face Recognition, Omkar M. Parkhi et al. (2015). Deep Face Recognition demonstrates the viability of large-scale web-scraped face datasets, providing the direct architectural and methodological precursor for the MS-Celeb-1M benchmark.
Cover for MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition

Abstract

In this paper, we design a benchmark task and provide the associated datasets for recognizing face images and link them to corresponding entity keys in a knowledge base. More specifically, we propose a benchmark task to recognize one million celebrities from their face images, by using all the possibly collected face images of this individual on the web as training data. The rich information provided by the knowledge base helps to conduct disambiguation and improve the recognition accuracy, and contributes to various real-world applications, such as image captioning and news video analysis. Associated with this task, we design and provide concrete measurement set, evaluation protocol, as well as training data. We also present in details our experiment setup and report promising baseline results. Our benchmark task could lead to one of the largest classification problems in computer vision. To the best of our knowledge, our training dataset, which contains 10M images in version 1, is the largest publicly available one in the world.

Topics

classificationcomputer visionfacial imagesknowledge basehigh resolution imageface featuresfrontal facesclassification models

Keywords

FaceImagesRecognitionCelebritiesDatasetFace RecognitionImageSetBenchmarkTraining

Table of Contents

  • 1 Introduction
  • Contribution Summary
  • 2 Related works
  • 3 Benchmark construction
  • 3.1 One million celebrity list
  • 3.2 Celebrity selection for measurement
  • 3.3 Labeling for measurement
  • Scraping
  • 4 Celebrity recognition
  • 4.1 Evaluation Protocol
  • Setup
  • Evaluation metric
  • 4.2 Training dataset
  • 4.3 Baseline
  • 5 Discussion and Future work
  • References

Knowls

  1. Knowl 1 — Knowledge-Base Linked Face Recognition

    definition

    Knowledge-base linked face recognition is the task of determining the identity of an individual depicted in a face image by explicitly mapping the detected face to a unique entity identifier (such as a Freebase Machine Identifier, or MID) within a structured knowledge graph. Unlike conventional face verification or open-set face identification that outputs text name strings or pairwise similarity scores, linking to a knowledge base resolves entity disambiguation (e.g., distinguishing different people who share identical names) and grounds recognition in rich structured entity properties such as profession, nationality, and birth date.

  2. Knowl 2 — MS-Celeb-1M Training Dataset

    experimental setup

    The MS-Celeb-1M (version 1) training dataset contains approximately 10 million face images representing the top 100,000 celebrities ranked by web occurrence frequency from a 1-million celebrity entity list. For each celebrity entity, approximately 100 image candidates are gathered from web search engines using multiple query formulations (including entity name, name with profession, and multi-language name variants). The dataset provides original image thumbnails alongside aligned and cropped facial regions. Web search noise is intentionally retained to facilitate research on scalable, noise-tolerant deep representation learning.

  3. Knowl 3 — Sub-Linear Frequency Weighting for Long-Tail Measurement Celebrity Selection

    equation

    Let fif_i denote the number of web documents mentioning the ii-th celebrity in a candidate entity list. To construct an evaluation set that represents both high-frequency public figures and low-frequency long-tail figures, each entity is sampled with a probability proportional to an adjusted frequency weight fi′f'_i:

    fi′=fi1/5f'_i = f_i^{1/\sqrt{5}}

    Because the raw document frequency distribution fif_i is extremely long-tailed (over 90% of celebrities have fi<30f_i < 30, while the top celebrities exceed 10610^6 mentions), sampling directly with fif_i yields fewer than 10% of evaluation entities from the bottom 90% of the entity list. The power-law exponent 1/5≈0.4471/\sqrt{5} \approx 0.447 dampens popularity bias, ensuring that approximately 25% of the evaluation entities are sampled from the bottom 90% tail.

  4. Knowl 4 — Precision and Coverage Metrics under Distractor Rejection

    equation

    In a face recognition measurement set consisting of mm ground-truth labeled celebrity images {xi}i=1m\{x_i\}_{i=1}^m and n−mn-m distractor face images {xi}i=m+1n\{x_i\}_{i=m+1}^n, a recognition model outputs a predicted entity key g^(xi)\hat{g}(x_i) and a confidence score c(xi)∈Rc(x_i) \in \mathbb{R} for each image xix_i, with ground-truth entity key g(xi)g(x_i). Predictions with confidence below a threshold tt are rejected. Precision P(t)P(t) is defined over non-rejected labeled images as:

    P(t)=∣{xi∣g^(xi)=g(xi)∧c(xi)≥t, i∈{1,…,m}}∣∣{xi∣c(xi)≥t, i∈{1,…,m}}∣P(t) = \frac{\left|\left\{x_i \mid \hat{g}(x_i) = g(x_i) \land c(x_i) \ge t, \, i \in \{1, \dots, m\}\right\}\right|}{\left|\left\{x_i \mid c(x_i) \ge t, \, i \in \{1, \dots, m\}\right\}\right|}

    and coverage C(t)C(t) is defined as the fraction of total labeled images that are recognized with confidence meeting the threshold:

    C(t)=∣{xi∣c(xi)≥t, i∈{1,…,m}}∣mC(t) = \frac{\left|\left\{x_i \mid c(x_i) \ge t, \, i \in \{1, \dots, m\}\right\}\right|}{m}

    The primary benchmark performance metric is the maximum coverage C(t)C(t) achieved subject to the constraint P(t)≥Pmin⁡P(t) \ge P_{\min}, where Pmin⁡=0.95P_{\min} = 0.95 (or 0.990.99).

  5. Knowl 5 — Two-Tier Measurement Set Benchmark Protocol

    experimental setup

    The MS-Celeb-1M measurement benchmark uses a sampled set of 1,500 celebrities (500 published as a development set, and 1,000 held out for evaluation). Candidate images are harvested across query variations and manually verified by multiple judges inspecting original web pages. For each celebrity, exactly two evaluation images are selected:

    • Random Set: One image chosen at random from the verified pool per celebrity to assess baseline entity coverage.
    • Hard Set: The verified image that exhibits the maximum visual appearance difference from any image of that celebrity in the training set, evaluating generalization across extreme intra-class variations (such as lighting, makeup, pose, and aging).

    Both subsets are blended with distractor images of other people in a randomized order to prevent manual identification of evaluation targets.

  6. Knowl 6 — Staged Curriculum Fine-Tuning for 100K-Class Face Recognition

    model/method

    Directly training a deep convolutional neural network from scratch on a classification objective with 100,000 output classes fails to converge due to the extreme category cardinality. To overcome this optimization barrier, training is conducted in progressive stages:

    1. Initial Fine-Tuning: An AlexNet-style deep convolutional neural network is initialized from generic pre-trained weights and trained for 50,000 iterations on a 500-class subset corresponding to the 500 celebrities with the largest numbers of images.
    2. Full 100K-Class Training: The learned weights from the 500-class network are used to initialize the 100,000-class network. The full network is trained for 250,000 iterations with an initial learning rate of 0.010.01, reduced by a factor of 10 to 0.0010.001 after 100,000 iterations, and to 0.00010.0001 after 200,000 iterations.

  7. Knowl 7 — Baseline Recognition Performance on MS-Celeb-1M Development Set

    empirical result

    A 100,000-class deep convolutional neural network trained via staged fine-tuning on the MS-Celeb-1M training set achieves the following coverage results on the 500-celebrity development set mixed with distractors:

    Evaluation Set Coverage@Precision 99% Coverage@Precision 95%
    Hard Set 0.052 0.442
    Random Set 0.606 0.728

    On the Random Set, coverage reaches 0.728 (72.8%) at 95% precision, approaching the 75% coverage upper bound dictated by the 100K training set. On the Hard Set, which tests robustness to appearance divergence, coverage drops to 0.442 (44.2%) at 95% precision and 0.052 (5.2%) at 99% precision.

  8. Knowl 8 — Freebase-Derived One Million Celebrity List Construction

    experimental setup

    The 1-million celebrity entity universe is curated from Freebase (which contains over 50 million topics and 2 billion triplets) by filtering for real human beings with defined facial appearances and public attention, restricted to individuals born after 1846 (the approximate era of commercial roll-film photography). The selected entities are ranked by web document occurrence frequency. The resulting 1M entity list covers over 2,000 distinct professions, more than 200 countries and regions, wide age variations, and three major racial groups (Caucasian, Mongoloid, Negroid).

  9. Knowl 9 — Training Set Coverage Ceiling on MS-Celeb-1M Benchmark

    limitation

    Because the initial (version 1) MS-Celeb-1M training dataset covers only the top 100,000 celebrities while the measurement set is sampled from a universe of 1,000,000 celebrities with an intentional 25% tail allocation, the training set includes approximately 75% of the evaluation entities. Consequently, any classification model trained exclusively on the provided v1 training data has a theoretical recall/coverage ceiling of approximately 75% on the measurement set.

Coverage note — None was omitted; all key contributions including task formulation, dataset generation, long-tail sampling formulas, evaluation metrics, staged baseline training, empirical results, and dataset limitations are fully represented.

References

  1. 1.Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: challenge of recognizing one million celebrities in the real world. In: IS&T International Symposium on Electronic Imaging (2016)
  2. 2.Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), June 2014
  3. 3.Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Web-scale training for face identification. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2746–2754. IEEE (2015)
  4. 4.Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), June 2015
  5. 5.Google: Freebase data dumps (2015). https://developers.google.com/freebase/data
  6. 6.Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015)
  7. 7.Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07–49, University of Massachusetts, Amherst, October 2007
  8. 8.Huang, G.B., Learned-Miller, E.: Labeled faces in the wild: updates and new reporting procedures. Technical report UM-CS-2014-003, University of Massachusetts, Amherst, May 2014
  9. 9.Sun, Y., Wang, X., Tang, X.: DeepID3: face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873 (2014)
  10. 10.Fan, H., Yang, M., Cao, Z., Jiang, Y., Yin, Q.: Learning compact face representation: packing a face into an int32. In: Proceedings of ACM International Conference on Multimedia, pp. 933–936. ACM (2014)
  11. 11.Kemelmacher-Shlizerman, I., Seitz, S., Miller, D., Brossard, E.: The MegaFace Benchmark: 1 Million Faces for Recognition at Scale. ArXiv e-prints (2015)
  12. 12.Ng, H.W., Winkler, S.: A data-driven approach to cleaning large face datasets. In: Proceedings of IEEE International Conference on Image Processing (ICIP), October 2014
  13. 13.Panis, G., Lanitis, A.: An overview of research activities in facial age estimation using the FG-NET aging database. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014 Workshops. LNCS, vol. 8926, pp. 737–750. Springer, Heidelberg (2015)
  14. 14.Wolf, L., Hassner, T., Maoz, I.: Face recognition in unconstrained videos with matched background similarity. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2011)
  15. 15.Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), June 2014
  16. 16.Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. arXiv preprint arXiv:1411.7923 (2014)
  17. 17.Klare, B.F., Klein, B., Taborsky, E., Blanton, A., Cheney, J., Allen, K., Grother, P., Mah, A., Jain, A.K.: Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus benchmark A. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), June 2015
  18. 18.Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings of the British Machine Vision Conference (BMVC) (2015)
  19. 19.Eastman, G.: Camera. US Patent 388850 A (1888)
  20. 20.Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 1097–1105. MIT Press (2012)
  21. 21.Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems (NIPS), pp. 849–856. MIT Press (2001)
  22. 22.Belkin, M., Niyogi, P.: Semi-supervised learning on Riemannian manifolds. J. Mach. Learn. 56(1–3), 209–239 (2004)
  23. 23.Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of International Conference on Machine Learning, pp. 912–919 (2003)
  24. 24.Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schlkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems (NIPS), pp. 321–328. MIT Press (2004)

Citation

MLA
Guo, Y., et al. “MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition”. Lecture Notes in Computer Science, Springer International Publishing, 2016, pp. 87–102, https://doi.org/10.1007/978-3-319-46487-9_6.
APA
Guo, Y., Zhang, L., Hu, Y., He, X., & Gao, J. (2016). MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. In Lecture Notes in Computer Science (pp. 87–102). Springer International Publishing. https://doi.org/10.1007/978-3-319-46487-9_6
Chicago
Guo, Y., L. Zhang, Y. Hu, X. He, and J. Gao. 2016. “MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition”. In Lecture Notes in Computer Science. Springer International Publishing. https://doi.org/10.1007/978-3-319-46487-9_6.
Harvard
Guo, Y. et al. (2016) “MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition”, Lecture Notes in Computer Science. Springer International Publishing, pp. 87–102. Available at: https://doi.org/10.1007/978-3-319-46487-9_6.
Vancouver
1. Guo Y, Zhang L, Hu Y, He X, Gao J (2016) MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. In: Lecture Notes in Computer Science. Springer International Publishing, pp 87–102

BibTeX

@inbook{Guo_2016, title={MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition}, ISBN={9783319464879}, ISSN={1611-3349}, url={http://dx.doi.org/10.1007/978-3-319-46487-9_6}, DOI={10.1007/978-3-319-46487-9_6}, booktitle={Computer Vision – ECCV 2016}, publisher={Springer International Publishing}, author={Guo, Yandong and Zhang, Lei and Hu, Yuxiao and He, Xiaodong and Gao, Jianfeng}, year={2016}, pages={87–102} }
Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF