The method of least squares
Era 1 · The foundations (1763–1942) › Probability and statistics
- Lineage
- Data and learning
- Kind
- Publication
Who
What happened
In Paris, Adrien-Marie Legendre publishes Nouvelles méthodes pour la détermination des orbites des comètes, with an appendix on the method of least squares.
Why it mattered
The first clear account of how to fit a model to measurements with errors: choose the values that make the sum of the squared errors as small as possible. It is the direct ancestor of regression and of training by minimizing an error function.
The full story
In the early nineteenth century, astronomers and geodesists faced a practical problem. They had more observations than unknowns, and since every measurement carried its own error, the equations would not all fit at once. They needed a principled way to choose a solution that spread those errors out.
In 1805 the French mathematician Adrien-Marie Legendre proposed one, in a few pages appended to a book on the orbits of comets: choose the values that minimize the sum of the squared errors. As Legendre explained it, this sets up "a kind of equilibrium" among the errors that keeps the extreme ones from carrying too much weight. The example used the same data on the shape of the Earth that Laplace had worked with.
The method had several parents. Carl Friedrich Gauss published it in 1809 and claimed to have used it since 1795, which set off a priority dispute with Legendre. In 1801, at 24, Gauss had applied it to calculate the orbit of Ceres, the asteroid Giuseppe Piazzi had lost sight of after 40 days of observation; thanks to Gauss's prediction, Franz Xaver von Zach found it again. In the United States, Robert Adrain reached the same idea in 1808.
Gauss later tied the method to the normal distribution of errors. Minimizing a sum of squared errors is still the most common way to fit a model to data, from linear regression to many neural networks.
Five scattered points and a line running between them; from each point grows a square whose side is the point's vertical distance to the line.
Under the hood
Given measurements yᵢ and a model that predicts f(xᵢ) from some parameters, the method picks the parameters that minimize the sum Σ (yᵢ − f(xᵢ))². Squaring keeps errors of opposite sign from canceling out and penalizes large errors more heavily.
If the model is linear in its parameters, like a straight line y = a + bx, the minimum comes from solving a system of linear equations, the "normal equations." It is the same calculation a linear regression performs today, and the same idea, minimizing a squared error, guides the training of many neural networks.
Leads to
Théorie analytique des probabilités (1812)
Inherited, with no proven direct link
Laplace's treatise folds the method of least squares into Laplace's general theory of probability
Sources
Original source
Further reading
How to cite
“The method of least squares.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/legendre-least-squares
This entry was last reviewed on November 13, 2026.