On the Opportunities and Risks of Foundation Models

Rishi BommasaniDrew A. HudsonEhsan AdeliRuss AltmanSimran AroraSydney von ArxMichael S. BernsteinJeannette BohgAntoine BosselutEmma Brunskill

article2021arXiv7,355 citations

Defines the paradigm of foundation models to systematically examine how large-scale pretrained systems yield emergent capabilities while concentrating technical, legal, and societal risks across downstream applications.

Listen

Artificial intelligence is undergoing a significant transformation marked by the rapid adoption of large-scale models trained on broad data via self-supervised learning, which are then adapted for a wide variety of downstream tasks. While these systems deliver remarkable capabilities across industries such as healthcare, law, and education, their rapid deployment introduces serious operational, ethical, and systemic risks that organizations must actively manage.

The article establishes a comprehensive framework to examine the dual characteristics of these systemstermed foundation modelsby analyzing their core capabilities, technical principles, practical applications, and broad societal impacts. The authors conducted an extensive interdisciplinary review synthesizing progress across deep learning, transfer learning, natural language processing, computer vision, and robotics, evaluating both empirical advancements and sociotechnical challenges.

The article identifies five primary findings. First, foundation models are defined by two interacting forces: emergence, where advanced behaviors arise implicitly rather than from explicit programming, and homogenization, where a single base model powers numerous downstream applications. Second, while homogenization creates massive leverage by standardizing development, it introduces severe single points of failure, meaning any flaw, security vulnerability, or bias in the base model is automatically inherited across all downstream deployments. Third, scaling up parameters and training data yields emergent abilities, such as in-context learning and zero-shot task transfer, but also makes the internal mechanics of these systems opaque and prone to unexpected failure modes. Fourth, training these models requires massive computational resources, which concentrates development within a few well-resourced industrial technology firms and restricts academic access. Fifth, applying these models to high-stakes domains requires high sample efficiency, yet current systems still struggle with complex multimodal grounding, physical-world safety constraints, and reliable long-form factual generation.

These findings indicate that treating foundation models purely as isolated software components is inadequate for managing enterprise and societal risk. System defects directly affect cost, legal compliance, safety, and brand reputation across entire product ecosystems. Furthermore, reliance on proprietary, resource-intensive base models shifts control over critical digital infrastructure to a small group of commercial entities, requiring leaders to re-evaluate vendor dependencies, liability frameworks, and auditing procedures.

To safely capitalize on these technologies, organizations should implement rigorous downstream auditing, data governance, and application-specific safety guardrails rather than deploying raw models directly. Decision-makers must invest in surrogate evaluation metrics, promote transparent documentation (such as model and data sheets), and advocate for public computational infrastructure to preserve open, reproducible research. When deciding between using existing base models or building custom adaptations, leaders must carefully weigh performance gains against the trade-offs of inherited systemic risk, carbon footprint, and ongoing fine-tuning costs.

The article acknowledges key limitations, including the rapidly evolving nature of the technology and the scarcity of established mathematical theory to explain how foundation models operate and when they fail. Confidence in their demonstrated empirical capabilities is high, but stakeholders should exercise caution regarding deployment safety, factual reliability, and ethical compliance until robust evaluation standards and governance frameworks are established.

arXiv: 2108.07258
  • Paper: Language Models are Few-Shot Learners, T. B. Brown et al. (2020). Reading GPT-3's introduction to few-shot scaling provides essential context for understanding the capabilities and prompt-based adaptation of foundation models analyzed in the source.
  • Paper: On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜, Emily M. Bender et al. (2021). Examining the social and ethical dangers of stochastic parrots offers critical background on the societal risks and uncurated web data dependencies discussed in the source report.
  • Paper: Model Cards for Model Reporting, Margaret Mitchell et al. (2019). Reviewing model cards for reporting provides foundational guidance on documenting performance metrics and demographic biases that underpin the risk analysis of foundation models.
  • Paper: Shortcut learning in deep neural networks, Robert Geirhos et al. (2020). Understanding shortcut learning in deep neural networks clarifies why large-scale models fail under real-world shifts, directly informing the risk and evaluation sections of the source.
Cover for On the Opportunities and Risks of Foundation Models

Abstract

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

Citation

MLA
Bommasani, R., et al. “On the Opportunities and Risks of Foundation Models”. arXiv, 2021, http://arxiv.org/abs/2108.07258v3.
APA
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., Arx, S. von ., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N., Chen, A., Creel, K., Davis, J. Q., Demszky, D., … Liang, P. (2021). On the Opportunities and Risks of Foundation Models. arXiv. http://arxiv.org/abs/2108.07258v3
Chicago
Bommasani, R., D. A. Hudson, E. Adeli, et al. 2021. “On the Opportunities and Risks of Foundation Models”. arXiv. http://arxiv.org/abs/2108.07258v3.
Harvard
Bommasani, R. et al. (2021) “On the Opportunities and Risks of Foundation Models”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2108.07258v3.
Vancouver
1. Bommasani R, Hudson DA, Adeli E, et al (2021) On the Opportunities and Risks of Foundation Models. arXiv

BibTeX

@article{bommasani2021the,
  title = {On the Opportunities and Risks of Foundation Models},
  author = {Bommasani, Rishi and Hudson, Drew A. and Adeli, Ehsan and Altman, Russ and Arora, Simran and Arx, Sydney von and Bernstein, Michael S. and Bohg, Jeannette and Bosselut, Antoine and Brunskill, Emma and Brynjolfsson, Erik and Buch, Shyamal and Card, Dallas and Castellon, Rodrigo and Chatterji, Niladri and Chen, Annie and Creel, Kathleen and Davis, Jared Quincy and Demszky, Dora and Donahue, Chris and Doumbouya, Moussa and Durmus, Esin and Ermon, Stefano and Etchemendy, John and Ethayarajh, Kawin and Fei-Fei, Li and Finn, Chelsea and Gale, Trevor and Gillespie, Lauren and Goel, Karan and Goodman, Noah and Grossman, Shelby and Guha, Neel and Hashimoto, Tatsunori and Henderson, Peter and Hewitt, John and Ho, Daniel E. and Hong, Jenny and Hsu, Kyle and Huang, Jing and Icard, Thomas and Jain, Saahil and Jurafsky, Dan and Kalluri, Pratyusha and Karamcheti, Siddharth and Keeling, Geoff and Khani, Fereshte and Khattab, Omar and Koh, Pang Wei and Krass, Mark and Krishna, Ranjay and Kuditipudi, Rohith and Kumar, Ananya and Ladhak, Faisal and Lee, Mina and Lee, Tony and Leskovec, Jure and Levent, Isabelle and Li, Xiang Lisa and Li, Xuechen and Ma, Tengyu and Malik, Ali and Manning, Christopher D. and Mirchandani, Suvir and Mitchell, Eric and Munyikwa, Zanele and Nair, Suraj and Narayan, Avanika and Narayanan, Deepak and Newman, Ben and Nie, Allen and Niebles, Juan Carlos and Nilforoshan, Hamed and Nyarko, Julian and Ogut, Giray and Orr, Laurel and Papadimitriou, Isabel and Park, Joon Sung and Piech, Chris and Portelance, Eva and Potts, Christopher and Raghunathan, Aditi and Reich, Rob and Ren, Hongyu and Rong, Frieda and Roohani, Yusuf and Ruiz, Camilo and Ryan, Jack and Ré, Christopher and Sadigh, Dorsa and Sagawa, Shiori and Santhanam, Keshav and Shih, Andy and Srinivasan, Krishnan and Tamkin, Alex and Taori, Rohan and Thomas, Armin W. and Tramèr, Florian and Wang, Rose E. and Wang, William and Wu, Bohan and Wu, Jiajun and Wu, Yuhuai and Xie, Sang Michael and Yasunaga, Michihiro and You, Jiaxuan and Zaharia, Matei and Zhang, Michael and Zhang, Tianyi and Zhang, Xikun and Zhang, Yuhui and Zheng, Lucia and Zhou, Kaitlyn and Liang, Percy},
  year = {2021},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2108.07258v3},
  eprint = {2108.07258}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/