Reinforcement Learning: An Introduction
Era 3 · From the expert boom to the statistical turn (1980–2011) › Reinforcement learning
- Lineage
- Data and learning
- Kind
- Publication
Who
University of Massachusetts Amherst, MIT Press
What happened
Richard Sutton and Andrew Barto publish Reinforcement Learning: An Introduction with MIT Press, the first complete textbook in the field.
Why it mattered
It brings dynamic programming, Monte Carlo methods, and temporal differences together in a common framework. It is still the standard reference in reinforcement learning.
The full story
In the 1990s reinforcement learning was a young, scattered field. Its ideas came from the psychology of animal learning, from optimal control theory, and from artificial intelligence, each with its own vocabulary and with results spread across the journals of each discipline.
Barto, who had arrived at the University of Massachusetts Amherst in 1977, and Sutton, Barto's former doctoral student, had spent two decades building that field. Their book organized it around a few ideas: an agent that interacts with an environment, rewards, value functions that estimate future reward, and policies that say what to do. On that framework they presented three families of methods, dynamic programming, Monte Carlo methods, and temporal differences, and showed how they combine.
The book also included a chapter of history that tied the threads together: animal conditioning, Bellman's work, Samuel's checkers program, and the authors' own contributions. That genealogy is the one still told today.
An expanded second edition came out in 2018, and both are available online. When the ACM gave Sutton and Barto the 2024 Turing Award, announced in March 2025, it called the book the standard reference in the field and noted that it has been cited more than 75,000 times.
Sources
Original source
- Reinforcement Learning: An Introduction, 1st edition, MIT Press. Section "History of Reinforcement Learning" of the online edition (opens in a new tab). Richard S. Sutton and Andrew G. Barto, 1998
Scholarship and context
- Catalog record for Reinforcement Learning: An Introduction (University of Texas at Austin) (opens in a new tab)
- ACM A.M. Turing Award Honors Two Researchers Who Led the Development of Cornerstone AI Technology (ACM press release, March 5, 2025) (opens in a new tab). Association for Computing Machinery, 2025
How to cite
“Reinforcement Learning: An Introduction.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/sutton-barto-textbook
This entry was last reviewed on November 13, 2026.