The History of AI

Chain of thought

Era 5 · Generative AI, reasoning, and agents (2022–2026) › Alignment and reasoning

Lineage
Symbols and logic
Kind
Publication

Who

Google Brain

What happened

Jason Wei and colleagues at Google show that if a large language model is given a few examples worked out with the intermediate steps of the reasoning, the model writes out those steps too and gets far more problems right in arithmetic, commonsense, and logic.

Why it mattered

Asking the model for the intermediate steps improves its reasoning, and the improvement appears only in sufficiently large models. It is the seed of the models that think before they answer.

The full story

In early 2022 large language models wrote fluently but failed at problems that take several steps, such as a grade-school math word problem. The usual way to prompt them with examples was to show them questions and final answers.

Wei and colleagues changed only the examples. Instead of the bare answer, each example included the chain of thought that leads to it, written in natural language: first this, then that, therefore the result. Faced with a new question, the model imitated the format, wrote out its own steps, and reached the right answer far more often.

They tested the technique on three families of models and on arithmetic, commonsense, and symbolic reasoning tasks. The most striking result was that a 540-billion-parameter model, with only eight such examples, reached the best published accuracy on GSM8K, a set of math problems, beating even a GPT-3 fine-tuned for the task and paired with a verifier.

The authors observed that the ability emerged only beyond a certain model size. The idea that reasoning in writing improves answers was the starting point for techniques such as tree of thoughts and, two years later, for models trained to reason, such as o1.

Under the hood

The technique does not change the model, only the prompt. A standard example has the form question, answer. One with a chain of thought has the form question, steps, answer, and the model continues that pattern with the new question.

With PaLM, at 540 billion parameters, and eight such examples, accuracy on GSM8K went from around 18 percent with standard examples to 57 percent. The previous best, 55 percent, belonged to a GPT-3 fine-tuned for the task and paired with a verifier.

The effect depends on size. In models below roughly 100 billion parameters, writing out the steps did not help and could even make the answer worse: the steps the model wrote were fluent but illogical.

Videos

Builds on

  • GPT-3 (May 28, 2020)

    Documented direct influence

    GPT-3 is one of the three model families the paper tests chain of thought on, and the baseline it beats on GSM8K is GPT-3 fine-tuned with a verifier (abstract and section 3)

Leads to

  • ReAct (October 6, 2022)

    Documented direct influence

    ReAct starts from chain-of-thought reasoning, which it cites as an example of reasoning cut off from the outside world, and interleaves it with actions (abstract)

  • Tree of thoughts (May 17, 2023)

    Documented direct influence

    Tree of thoughts "generalizes over the popular Chain of Thought approach" (abstract)

  • OpenAI o1 (September 12, 2024)

    Inherited, with no proven direct link

    o1 is trained with reinforcement learning to make productive use of a chain of thought before answering

Threads that pass through here

Sources

Original source

How to cite

“Chain of thought.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/chain-of-thought

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline