The History of AI

LeNet-5 and MNIST

Era 3 · From the expert boom to the statistical turn (1980–2011) › Neural networks

Lineage
Neurons and networks
Kind
Implementation

Who

AT&T Labs

What happened

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner publish "Gradient-based learning applied to document recognition" in the Proceedings of the IEEE, presenting the LeNet-5 convolutional network and the MNIST database of handwritten digits, built before 1994.

Why it mattered

LeNet-5 is the classic convolutional network, and the check-reading system built on it was used commercially. MNIST becomes the standard benchmark for comparing image recognition methods.

The full story

Ten years after the zip code network, LeCun's group published a long paper that gathered everything it had learned. LeNet-5 alternates convolutional and subsampling layers, with about 60,000 trainable parameters, and ends in fully connected layers that decide which digit it is looking at. The paper compared many methods in detail on the same dataset and showed that gradient-based learning beat systems built on hand-designed features.

The dataset was MNIST, which LeCun, Corinna Cortes, and Christopher Burges had assembled by mixing two databases from the US National Institute of Standards and Technology, one written by Census Bureau employees and the other by students. It has 60,000 training images and 10,000 test images, each 28 by 28 pixels, centered and normalized.

The paper also described a complete check-reading system, which combined the network with a model of character sequences and was in commercial use.

For two decades MNIST was machine learning's default test bed: almost every new method was introduced with its error rate on MNIST. Today it is considered too easy, but it is still the first example used to teach how to train a network.

Builds on

Leads to

  • DanNet (2011)

    Documented direct influence

    DanNet is a LeNet-style convolutional network, deeper and trained on GPUs; the paper cites the 1998 work by LeCun et al.

Threads that pass through here

Sources

How to cite

“LeNet-5 and MNIST.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/lenet-5-and-mnist

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline