The Hopfield network
Era 3 · From the expert boom to the statistical turn (1980–2011) › Neural networks
- Lineage
- Neurons and networks
- Kind
- Publication
Who
Caltech
What happened
The physicist John Hopfield publishes "Neural networks and physical systems with emergent collective computational abilities" in the Proceedings of the National Academy of Sciences.
Why it mattered
A recurrent network works as an associative memory: it stores patterns and retrieves them from an incomplete or noisy version. It brings the physics of magnetic materials to networks and draws physicists back into the field.
The full story
Hopfield, a physicist by training, asked how a collection of simple elements could produce collective properties like memory. The model was spin glasses, disordered magnetic materials in which some pairs of neighboring atoms tend to align their orientation and others to oppose it, so the material has many different stable states.
In the network, each neuron is either active or inactive and is connected to every other neuron. The connections are set with a Hebb-type rule from the patterns to be stored. Hopfield defined an energy for each state of the network and proved that every time a neuron updates, the energy goes down or stays the same. The network therefore falls toward energy minima, and the stored patterns are those minima. Shown an incomplete image, the network slides to the most similar stored pattern: it remembers by association, the way a snatch of a song brings back the rest.
Amari in 1972 and William Little in 1974 had proposed similar models, and Hopfield cited Little. But the clear presentation, tied to statistical physics, drew many physicists into neural networks at a time when AI had set them aside. In 1984 Hopfield extended the model to neurons with continuous outputs.
In 2024 Hopfield received the Nobel Prize in Physics, together with Geoffrey Hinton, for the discoveries that made machine learning with neural networks possible.
A wavy energy curve over an axis of network states, with two valleys marked memory A and memory B. A ball on a slope, the incomplete cue, has an arrow carrying it to the bottom of memory B's valley.
Under the hood
In the usual form, each neuron sᵢ is +1 or −1. To store several patterns ξ, the weights are set with a Hebb rule, wᵢⱼ = (1/N)·Σ ξᵢ·ξⱼ summed over the patterns, with symmetric weights and no connections from a neuron to itself. The energy of a state is E = −½·Σ wᵢⱼ·sᵢ·sⱼ.
If the neurons update one at a time with sᵢ ← sign(Σⱼ wᵢⱼ·sⱼ), the energy never rises, and since the number of states is finite, the network ends up at rest in a minimum. The stored patterns are minima of that function, which is why the network recovers them from a noisy version.
Memory has a limit. With the Hebb rule, a network of N neurons reliably recovers about 0.14·N random patterns. Beyond that, spurious minima appear, blends of memories, and the network stops remembering.
Videos
Leads to
The Boltzmann machine (January 1985)
Documented direct influence
The Nobel committee explains that Hinton used the Hopfield network as the basis for a new network, the Boltzmann machine, with tools from statistical physics
The Boltzmann machine (January 1985)
Documented direct influence
Hinton builds on the Hopfield network to create the Boltzmann machine, with tools from statistical physics
Nobel Prize in Physics for Hopfield and Hinton (October 8, 2024)
Documented direct influence
The Academy's announcement names the Hopfield network and the Boltzmann machine as the work being honored
Threads that pass through here
- The neural network winter (step 1 of 11)
- From Hopfield's memory to the Nobel (step 1 of 3)
Sources
Original source
Scholarship and context
Further reading
How to cite
“The Hopfield network.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/hopfield-network
This entry was last reviewed on November 13, 2026.