LeCun's convolutional network
Era 3 · From the expert boom to the statistical turn (1980–2011) › Neural networks
- Lineage
- Neurons and networks
- Kind
- Application
Who
Bell Labs
What happened
Yann LeCun and colleagues at Bell Labs publish "Backpropagation Applied to Handwritten Zip Code Recognition" in Neural Computation: a convolutional network trained with backpropagation reads the handwritten zip codes of the US Postal Service.
Why it mattered
It joins the neocognitron's architecture to backpropagation. The network learns on its own the filters that detect features, and convolution becomes the central tool of computer vision.
The full story
Fukushima's neocognitron already alternated layers that detect local features with layers that tolerate shifts, but it learned without supervision and with ad hoc rules. LeCun, who had spent time in Hinton's group in Toronto before arriving at Bell Labs, proposed training the whole architecture end to end with backpropagation. In 1987, at the ATR lab in Japan, Alex Waibel and colleagues had trained a network with weights shared across time in this way to recognize phonemes; LeCun carried the idea over to images.
The central idea is weight sharing. In a convolutional layer, the same small detector is applied at every position in the image, so a feature that is useful in one corner is useful in the center too. That drastically reduces the number of parameters and lets the network learn which detectors it needs instead of having an engineer design them.
The 1989 network was trained on a few thousand zip code digits scanned from real mail and reached a low error rate on digits it had not seen. It was a practical application, not a toy example: the same team later took the technique to automated check reading.
In awarding LeCun the 2018 Turing Award, announced in 2019, the ACM highlighted that LeCun was the first to train a convolutional network on images of handwritten digits. Convolutional networks, scaled up and trained on GPUs, would dominate computer vision from 2012 on.
A 6 × 6 grid representing the image, with a 3 × 3 square highlighted in the top left corner and a dashed one in the bottom right. Lines join the square to a 3 × 3 filter, the same across the whole image, and the filter to one cell of a 4 × 4 feature map.
Under the hood
A convolutional layer applies the same filter at every position in the image. With an n × n image and a k × k filter, the result is a map of (n − k + 1) × (n − k + 1) responses, each the sum of the k² pixels in the window multiplied by the filter's weights.
Sharing the weights has two effects. There are far fewer parameters: a 5 × 5 filter has 25 weights and a bias, whatever the size of the image, whereas without sharing, each position on the map would have its own 25. And the feature is detected the same way anywhere: if the stroke shifts, its response shifts with it on the map.
Between convolutional layers the resolution is reduced, first by subsampling and later by taking the maximum of each window, so that higher layers see ever larger regions and tolerate small distortions. Backpropagation adjusts the filters like any other weight: nobody designs them by hand.
Videos
Builds on
The neocognitron (October 1979)
Inherited, with no proven direct link
The same hierarchical architecture; LeCun adds training by backpropagation
Backpropagation by Rumelhart, Hinton, and Williams (October 1986)
Documented direct influence
The 1989 network is trained with backpropagation, as the paper's title says
Leads to
LeNet-5 and MNIST (1998)
Documented direct influence
LeNet-5 is the 1989 convolutional network developed further, in the same group
Threads that pass through here
- The neural network winter (step 4 of 11)
- From the retina to the convolutional network (step 3 of 3)
Sources
Original source
- Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1, pp. 541-551 (opens in a new tab). Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard, and Lawrence Jackel, 1989
- Phoneme recognition using time-delay neural networks. IEEE Transactions on Acoustics, Speech, and Signal Processing, 37(3), pp. 328-339 (opens in a new tab). Alex Waibel, Toshiyuki Hanazawa, Geoffrey Hinton, Kiyohiro Shikano, and Kevin Lang, 1989
Scholarship and context
Further reading
How to cite
“LeCun's convolutional network.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/lecun-convolutional-network
This entry was last reviewed on November 13, 2026.