The History of AI

The Hopfield network

Era 3 · From the expert boom to the statistical turn (1980–2011) › Neural networks

Lineage
Neurons and networks
Kind
Publication

Who

Caltech

What happened

The physicist John Hopfield publishes "Neural networks and physical systems with emergent collective computational abilities" in the Proceedings of the National Academy of Sciences.

Why it mattered

A recurrent network works as an associative memory: it stores patterns and retrieves them from an incomplete or noisy version. It brings the physics of magnetic materials to networks and draws physicists back into the field.

The full story

Hopfield, a physicist by training, asked how a collection of simple elements could produce collective properties like memory. The model was spin glasses, disordered magnetic materials in which some pairs of neighboring atoms tend to align their orientation and others to oppose it, so the material has many different stable states.

In the network, each neuron is either active or inactive and is connected to every other neuron. The connections are set with a Hebb-type rule from the patterns to be stored. Hopfield defined an energy for each state of the network and proved that every time a neuron updates, the energy goes down or stays the same. The network therefore falls toward energy minima, and the stored patterns are those minima. Shown an incomplete image, the network slides to the most similar stored pattern: it remembers by association, the way a snatch of a song brings back the rest.

Amari in 1972 and William Little in 1974 had proposed similar models, and Hopfield cited Little. But the clear presentation, tied to statistical physics, drew many physicists into neural networks at a time when AI had set them aside. In 1984 Hopfield extended the model to neurons with continuous outputs.

In 2024 Hopfield received the Nobel Prize in Physics, together with Geoffrey Hinton, for the discoveries that made machine learning with neural networks possible.

A wavy energy curve over an axis of network states, with two valleys marked memory A and memory B. A ball on a slope, the incomplete cue, has an arrow carrying it to the bottom of memory B's valley.

Each stored memory is an energy valley. If the network starts from an incomplete cue, it rolls downhill to the nearest memory.

Under the hood

In the usual form, each neuron sᵢ is +1 or −1. To store several patterns ξ, the weights are set with a Hebb rule, wᵢⱼ = (1/N)·Σ ξᵢ·ξⱼ summed over the patterns, with symmetric weights and no connections from a neuron to itself. The energy of a state is E = −½·Σ wᵢⱼ·sᵢ·sⱼ.

If the neurons update one at a time with sᵢ ← sign(Σⱼ wᵢⱼ·sⱼ), the energy never rises, and since the number of states is finite, the network ends up at rest in a minimum. The stored patterns are minima of that function, which is why the network recovers them from a noisy version.

Memory has a limit. With the Hebb rule, a network of N neurons reliably recovers about 0.14·N random patterns. Beyond that, spurious minima appear, blends of memories, and the network stops remembering.

Videos

Leads to

  • The Boltzmann machine (January 1985)

    Documented direct influence

    The Nobel committee explains that Hinton used the Hopfield network as the basis for a new network, the Boltzmann machine, with tools from statistical physics

  • The Boltzmann machine (January 1985)

    Documented direct influence

    Hinton builds on the Hopfield network to create the Boltzmann machine, with tools from statistical physics

  • Nobel Prize in Physics for Hopfield and Hinton (October 8, 2024)

    Documented direct influence

    The Academy's announcement names the Hopfield network and the Boltzmann machine as the work being honored

Threads that pass through here

Sources

How to cite

“The Hopfield network.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/hopfield-network

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline