Forerunners of backpropagation
Era 2 · AI is born (1943–1979) › Neural networks and learning
- Lineage
- Neurons and networks
- Kind
- Theory
Who
What happened
Henry Kelley publishes "Gradient Theory of Optimal Flight Paths" in the ARS Journal, a method for optimizing flight trajectories. Arthur Bryson and Stuart Dreyfus develop similar methods in the early 1960s.
Why it mattered
They compute the gradient of an objective by propagating derivatives backward through the system, from each stage to the one before. It is a forerunner of backpropagation, from optimal control theory.
The full story
The space race and missiles posed a concrete problem: choosing a vehicle's controls over time so that it reaches its destination at minimum cost. The trajectory is a sequence of states, and each decision affects all the ones that follow.
Kelley proposed improving an initial trajectory step by step, in the direction of the gradient of the cost. Getting that gradient requires knowing how the final result changes when a control is nudged at any given moment, and the efficient way to compute it is to work from the end back to the beginning, chaining the derivatives of each stage to the next. Bryson and Dreyfus, by different routes, arrived at equivalent procedures.
That backward calculation is, in essence, what backpropagation does in a neural network, where the stages are layers instead of moments in time. But this work appeared in aerospace and control engineering journals, and at the time nobody connected it with the neural network research of Rosenblatt and Widrow, which needed exactly such a method.
Jürgen Schmidhuber's history of deep learning claims them as forerunners, though it notes that the efficient, general form of the method arrived in 1970 with Linnainmaa.
A chain of computation running left to right, from x to a multiplication by w, a function f, and the cost E. Below, dashed arrows run back from E to the left with the derivatives of the cost with respect to y, a, and w.
Leads to
Backpropagation by Rumelhart, Hinton, and Williams (October 1986)
Independent convergence
Rumelhart arrived at the method independently around 1982, and the 1986 paper does not cite the earlier work
Sources
Original source
Scholarship and context
- Annotated History of Modern AI and Deep Learning (arXiv:2212.11279) (opens in a new tab). Jürgen Schmidhuber, 2022
Further reading
How to cite
“Forerunners of backpropagation.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/kelley-bryson-optimal-control
This entry was last reviewed on November 13, 2026.