ADALINE
Era 2 · AI is born (1943–1979) › Neural networks and learning
- Lineage
- Neurons and networks
- Kind
- Implementation
Who
Stanford University
What happened
Bernard Widrow, a professor at Stanford, and a doctoral student, Marcian "Ted" Hoff, present ADALINE (Adaptive Linear Neuron) and its learning rule in the paper "Adaptive switching circuits."
Why it mattered
It adjusts the weights by gradient descent on the squared error, measured before the threshold is applied. The least mean squares (LMS) rule, or delta rule, is still used in adaptive filtering.
The full story
Widrow came from signal processing, not psychology, and framed learning as an optimization problem. The unit resembled the perceptron's, with weighted inputs, a bias, and a threshold, but with one decisive difference: the error was measured before the threshold, on the linear output. That makes the squared error a smooth function of the weights, and you can go down its slope.
The resulting least mean squares rule moves each weight in proportion to the error and to its input. It is a special case of gradient descent and takes very few operations to compute, which made it possible to build in hardware. The first ADALINE held its weights in rheostats turned by hand, and later ones in "memistors," electrochemical components Widrow designed to store adjustable weights.
MADALINE, with several units in two layers, followed in 1962, and in 1963 a machine with a thousand weights. But Widrow and Hoff did not find a general way to train the hidden layers either, and Widrow turned to adaptive filters.
That is where the rule had its greatest practical success: echo cancelers on telephone lines, modem equalizers, and adaptive antennas. Hoff went on to Intel and was one of the designers of the first microprocessor.
Side by side
| Aspect | Perceptron | ADALINE |
|---|---|---|
| Where the error is measured | After the threshold: right or wrong | Before the threshold, on the sum |
| When the weights are corrected | Only when it is wrong | Always, in proportion to the error |
| What the rule does | Adds or subtracts the input | Goes down the slope of the squared error |
| Where the weights were stored | Potentiometers turned by motors | Rheostats, then memistors |
Videos
Sources
Scholarship and context
- 30 years of adaptive neural networks: perceptron, Madaline, and backpropagation. Proceedings of the IEEE, 78(9), pp. 1415-1442 (opens in a new tab). Bernard Widrow and Michael A. Lehr, 1990
- Annotated History of Modern AI and Deep Learning (arXiv:2212.11279) (opens in a new tab). Jürgen Schmidhuber, 2022
Further reading
How to cite
“ADALINE.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/adaline
This entry was last reviewed on November 13, 2026.