The History of AI

TD-Gammon

Era 3 · From the expert boom to the statistical turn (1980–2011) › Reinforcement learning

Lineage
Data and learning
Kind
Implementation

Who

IBM

What happened

Gerald Tesauro of IBM's Thomas J. Watson Research Center develops TD-Gammon, a backgammon program with a neural network that learns by playing against itself using temporal differences.

Why it mattered

A network that learns by playing itself, with no expert knowledge at all in its first version, reaches a level close to that of the best human players and discovers moves that change the theory of the game.

The full story

Tesauro had earlier tried to teach backgammon to a neural network using games annotated by experts, and the result was a competent player but not a brilliant one. With TD-Gammon the method changed: the network played against itself, starting from random weights, and after each move adjusted its evaluation of the previous position using that of the next one, with Sutton's TD(λ).

At first the games were absurd, because both sides played badly. But the randomness of the dice forced the network to explore a wide variety of positions, and it kept improving. Later versions were also given board features designed by experts and a short search two or three moves deep. Version 2.1, from 1993, was trained on a million and a half games and played at a level slightly below the best humans. In a 100-game series in 1998 it lost to the world champion by only eight points.

Its effect on the game itself was striking. In some openings TD-Gammon preferred moves that experts considered inferior; players analyzed them, found that the program was right, and changed the way they played.

For twenty years TD-Gammon was the great example of reinforcement learning with neural networks. Its idea of a network that evaluates positions and improves by playing against itself reappears in AlphaGo and AlphaZero, which add tree search and learn from the final result of each game instead of from temporal differences.

Builds on

Leads to

  • AlphaGo beats Fan Hui (October 5, 2015)

    Inherited, with no proven direct link

    Both combine a neural network that evaluates positions with learning through self-play, using different training methods

Threads that pass through here

Sources

How to cite

“TD-Gammon.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/td-gammon

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline