The History of AI

Threads

25 threads that follow one idea in AI from start to finish, milestone by milestone, showing what led from each to the next.

  • From Boole to the circuit. The algebra Boole uses to write down logic is the one Shannon uses in 1937 to describe relay circuits.
  • From the neuron to the machine. Formal logic and the study of neural circuits meet in the McCulloch-Pitts neuron. Two years later, its notation turns up in the report that describes the stored program.
  • Shakey and its algorithms. A* and STRIPS were both created inside one robot project, Shakey.
  • From reward to alignment. From the idea that reward reinforces behavior to the technique used to fine-tune ChatGPT. Two lines run in parallel and meet in Sutton; two more meet in Ziegler's work.
  • Data, GPUs, and AlexNet. Programmable GPUs and a massive set of labeled images, the two missing ingredients, come together in AlexNet.
  • From attention to the Transformer. Attention first appears on top of a sequence-to-sequence model. The Transformer keeps the attention and drops the recurrence.
  • The neural network winter. How neural networks survived twenty years in the background, from Hopfield's physics to GPUs, and why they came back with AlexNet.
  • The rise and fall of expert systems. Japan's bet on logic programming sets off a race between countries for symbolic AI in the 1980s. Ten years later, the market has collapsed and the big projects end without delivering what they promised.
  • Machines versus champions. From Samuel's checkers to AlphaGo's Go, games measure how far AI has come. Two paths take turns, brute-force search and learning by self-play, until they merge.
  • Cars that drive themselves. The military money that drove the AI race in the 1980s pays for a test vehicle at Carnegie Mellon. A neural network learns to drive it, and two decades later an off-road vehicle crosses the desert with no one at the wheel.
  • From prompted reasoning to trained reasoning. A way of writing the examples in a prompt becomes a training method, and within two years every lab offers models that think before they answer.
  • From answering to acting. Language models go from writing answers to using tools, browsers, and computers, and to working on their own for hours.
  • The model that was too capable. In 2026 the most capable models find vulnerabilities better than almost anyone, and the three big American labs release them first, or only, to defenders.
  • China and the chips. The United States restricts sales of advanced chips to China, and Chinese labs respond with efficient open models and, later, with chips of their own.
  • From diffusion to world models. Diffusion models go from generating images to generating video and, later, interactive worlds that respond to whoever moves through them.
  • From Hopfield's memory to the Nobel. Two physics-inspired networks from the 1980s receive the 2024 Nobel Prize in Physics as foundations of machine learning.
  • From Hilbert's question to the universal machine. Hilbert asks for procedures that can settle any mathematical question. Gödel, Church, Post, and Turing prove that none exist and, to prove it, define what it means to compute.
  • The perceptron and its limits. The logical neuron and Hebb's rule come together in the perceptron, the first network that learns from examples. Minsky and Papert later map its limits.
  • From general reasoning to expert knowledge. Symbolic AI starts out looking for a general method of reasoning and ends up finding its strength in the specialized knowledge of expert systems.
  • From the retina to the convolutional network. The hierarchy of cells Hubel and Wiesel find in the cat's visual cortex becomes an artificial network and then convolutional networks.
  • From Atari to Go. In four years, DeepMind goes from learning video games from pixels to mastering Go without human games.
  • Scale as a method. From pretraining on unlabeled text to foundation models, by way of the laws that predict how much a model improves as it grows.
  • From noise to image. Three lines of work on generating images, latent space, diffusion, and denoising, come together in the image generators of 2022.
  • Who pays for compute. The cost of training ever larger models changes the shape of the organizations that build them.
  • From protein to Nobel. How a problem open for half a century, predicting the shape of a protein, is solved in two years and ends in the Nobel Prize in Chemistry.