The term "foundation model"
Era 4 · Deep learning and foundation models (2012–2021) › Transformers and language models
- Lineage
- Neurons and networks
- Kind
- Theory
Who
Stanford CRFM
What happened
On August 16, 2021, "On the Opportunities and Risks of Foundation Models" appears on arXiv, a report of more than 200 pages signed by 114 authors from Stanford's new Center for Research on Foundation Models. Stanford HAI announces the center and the report on the 18th.
Why it mattered
It coins "foundation model" for models trained on broad data that can be adapted to many tasks, such as BERT or GPT-3, and examines their opportunities and their risks side by side.
The full story
By 2021, a handful of large models such as BERT and GPT-3 were serving as the starting point for very different applications. This kind of model had no name yet, and the Stanford report proposed one: a foundation model is any model trained on broad data, generally with self-supervision at scale, that can be adapted to a wide range of tasks.
The authors explain the word. These models are central, because so much is built on top of them, and at the same time incomplete, because each use requires adapting them. The report frames the phenomenon around two ideas. Scale makes new capabilities appear that nobody programmed, and the models' effectiveness across so many tasks pushes everyone toward using the same model as the base for everything, so that its flaws are inherited by every model adapted from it.
The report covers capabilities (language, vision, robotics, reasoning), technical principles, applications (law, health care, education), and the effects on society: inequity, misuse, economic and environmental impact, and legal and ethical questions. It was released under a Creative Commons Attribution license, which allows it to be reproduced with credit to its authors.
The report went up on arXiv on August 16, 2021, and Stanford HAI announced the center and the report two days later.
Builds on
ELMo, GPT-1, and BERT (February 15, 2018), GPT-3 (May 28, 2020)
Documented direct influence
The report coins the term for models like BERT and GPT-3, which are trained on broad data and adapted to many tasks, and names them as examples from its first sentence
Threads that pass through here
- Scale as a method (step 7 of 8)
Sources
Original source
- On the Opportunities and Risks of Foundation Models (arXiv:2108.07258) (opens in a new tab). Rishi Bommasani and 113 others, Stanford Center for Research on Foundation Models, 2021. open license: CC BY 4.0
- p. 3 Introduction and Figure 1: the definition of a foundation model and the two ideas behind it, emergence and homogenization.
Scholarship and context
Further reading
How to cite
“The term "foundation model".” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/foundation-models
This entry was last reviewed on November 13, 2026.