The History of AI

Reinforcement Learning: An Introduction

Era 3 · From the expert boom to the statistical turn (1980–2011) › Reinforcement learning

Lineage
Data and learning
Kind
Publication

Who

University of Massachusetts Amherst, MIT Press

What happened

Richard Sutton and Andrew Barto publish Reinforcement Learning: An Introduction with MIT Press, the first complete textbook in the field.

Why it mattered

It brings dynamic programming, Monte Carlo methods, and temporal differences together in a common framework. It is still the standard reference in reinforcement learning.

The full story

In the 1990s reinforcement learning was a young, scattered field. Its ideas came from the psychology of animal learning, from optimal control theory, and from artificial intelligence, each with its own vocabulary and with results spread across the journals of each discipline.

Barto, who had arrived at the University of Massachusetts Amherst in 1977, and Sutton, Barto's former doctoral student, had spent two decades building that field. Their book organized it around a few ideas: an agent that interacts with an environment, rewards, value functions that estimate future reward, and policies that say what to do. On that framework they presented three families of methods, dynamic programming, Monte Carlo methods, and temporal differences, and showed how they combine.

The book also included a chapter of history that tied the threads together: animal conditioning, Bellman's work, Samuel's checkers program, and the authors' own contributions. That genealogy is the one still told today.

An expanded second edition came out in 2018, and both are available online. When the ACM gave Sutton and Barto the 2024 Turing Award, announced in March 2025, it called the book the standard reference in the field and noted that it has been cited more than 75,000 times.

Sources

How to cite

“Reinforcement Learning: An Introduction.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/sutton-barto-textbook

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline