LeNet-5 and MNIST
Era 3 · From the expert boom to the statistical turn (1980–2011) › Neural networks
- Lineage
- Neurons and networks
- Kind
- Implementation
Who
AT&T Labs
What happened
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner publish "Gradient-based learning applied to document recognition" in the Proceedings of the IEEE, presenting the LeNet-5 convolutional network and the MNIST database of handwritten digits, built before 1994.
Why it mattered
LeNet-5 is the classic convolutional network, and the check-reading system built on it was used commercially. MNIST becomes the standard benchmark for comparing image recognition methods.
The full story
Ten years after the zip code network, LeCun's group published a long paper that gathered everything it had learned. LeNet-5 alternates convolutional and subsampling layers, with about 60,000 trainable parameters, and ends in fully connected layers that decide which digit it is looking at. The paper compared many methods in detail on the same dataset and showed that gradient-based learning beat systems built on hand-designed features.
The dataset was MNIST, which LeCun, Corinna Cortes, and Christopher Burges had assembled by mixing two databases from the US National Institute of Standards and Technology, one written by Census Bureau employees and the other by students. It has 60,000 training images and 10,000 test images, each 28 by 28 pixels, centered and normalized.
The paper also described a complete check-reading system, which combined the network with a model of character sequences and was in commercial use.
For two decades MNIST was machine learning's default test bed: almost every new method was introduced with its error rate on MNIST. Today it is considered too easy, but it is still the first example used to teach how to train a network.
Builds on
LeCun's convolutional network (1989)
Documented direct influence
LeNet-5 is the 1989 convolutional network developed further, in the same group
Leads to
DanNet (2011)
Documented direct influence
DanNet is a LeNet-style convolutional network, deeper and trained on GPUs; the paper cites the 1998 work by LeCun et al.
Threads that pass through here
- The neural network winter (step 5 of 11)
Sources
Original source
- Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), pp. 2278-2324 (opens in a new tab). Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner, 1998
Scholarship and context
Further reading
How to cite
“LeNet-5 and MNIST.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/lenet-5-and-mnist
This entry was last reviewed on November 13, 2026.