Who pays for compute
The cost of training ever larger models changes the shape of the organizations that build them.
OpenAI is announced (December 11, 2015)
OpenAI is founded in 2015 as a nonprofit, with $1 billion pledged by its donors.
Reinforcement learning from human preferences (June 12, 2017)
Researchers at OpenAI and DeepMind train agents from people's preferences.
OpenAI adopts a capped-profit model (March 11, 2019)
In 2019 it creates a capped-profit company because it will need billions for compute, and Microsoft invests $1 billion and becomes its cloud provider.
Scaling laws (January 23, 2020)
The scaling laws predict how much a model improves when given more compute.
GPT-3 (May 28, 2020)
GPT-3 is trained on a Microsoft cluster, at a size justified by the scaling laws.
Anthropic is founded (January 2021)
Authors of GPT-3, the scaling laws, and learning from human preferences leave OpenAI and found Anthropic.