The History of AI

Glossary

A glossary of artificial intelligence: 86 terms explained from scratch, each linked to the milestone where it was born.

admissible heuristic (admissible heuristic)
An estimate of the remaining cost to the goal that never exceeds the real cost. With one, the A* algorithm always finds the shortest path. The A* algorithm
agent (agent)
A system that acts in an environment to achieve a goal. It observes, decides, acts, and observes again. In reinforcement learning, the agent is the learner; with language models, it is a model that uses tools and chains steps together without a person directing each one. ReAct. reinforcement learning
AI winter (AI winter)
A period in which interest in and funding for artificial intelligence collapse after a stretch of inflated expectations. There are said to have been two, in the 1970s and in the late 1980s. The "AI winter" warning
algorithm (algorithm)
A step-by-step procedure that, applied to some data, reaches a result in a finite number of steps without any need for ingenuity. In 1936, Church and Turing gave the first precise mathematical definitions of the idea. On Computable Numbers. Turing machine, lambda calculus
alignment (alignment)
The field that studies how to make an artificial intelligence system pursue the goals its designers and users actually want, and not a distorted version of them.
alpha-beta pruning (alpha-beta pruning)
A technique for searching a game tree that stops exploring branches that cannot change the decision, because the opponent would never allow play to reach them. Deep Blue beats Kasparov
attention mechanism (attention mechanism)
The part of a neural network that, when processing one element of a sequence, decides how much weight to give each of the others. That lets it connect words that are far apart. The attention mechanism. neural network
automatic differentiation (automatic differentiation)
Exact computation of a program's derivatives by applying the chain rule to each operation. In reverse mode, the computation is traversed from output to input and all the derivatives come out at once; backpropagation is a case of this mode. Reverse-mode automatic differentiation. backpropagation
backpropagation (backpropagation)
A method for training a multilayer neural network. It works out how much each weight contributes to the output error by traveling back through the network from output to input, and the weights are corrected accordingly. Backpropagation by Rumelhart, Hinton, and Williams. gradient descent, neural network
Bayes' theorem (Bayes' theorem)
A rule for updating the probability of a hypothesis when new data arrive, based on what was believed before and on how likely those data are if the hypothesis is true. Bayes's essay. Bayesian network, likelihood
Bayesian network (Bayesian network)
A graph in which each node is a variable and each arrow a direct dependency, with the probabilities of each variable given its causes. It allows reasoning under uncertainty. Bayesian networks
BLEU (bilingual evaluation understudy)
Bilingual evaluation understudy: an automatic measure of translation quality that counts how many word sequences match translations done by people. It runs from 0 to 100; the higher it is, the closer to a human translation.
Boolean algebra (Boolean algebra)
An algebra of two values, 0 and 1, with three operations: AND, OR, and NOT. It was born in the nineteenth century as a calculus of logic and underlies the design of every digital circuit. An Investigation of the Laws of Thought. relay
chain of thought (chain of thought)
The intermediate steps a language model writes out before giving its answer. At first they were elicited by asking for them with examples; since 2024, reasoning models have been trained to produce them. Chain of thought. reasoning model
combinatorial explosion (combinatorial explosion)
Growth in the number of possibilities, such as paths in a search, so fast as a problem gets longer that examining them all becomes impossible, even for a computer. General Problem Solver
conditioned reflex (conditioned reflex)
A response an animal learns to give to a signal because that signal has been presented several times together with a stimulus that already caused the response, like salivating at the sound that announces food. The conditioned reflex. law of effect
context window (context window)
The amount of text, measured in tokens, that a language model can take into account at once, counting both what it receives and what it writes. Whatever falls outside the window, the model does not see.. token, language model
convolutional network (convolutional neural network)
A neural network built for images. It applies the same small filters across the whole image, so it recognizes a feature wherever it appears. LeCun's convolutional network. neural network, deep learning
decision tree (decision tree)
A model that classifies by asking one question after another about the data, such as whether a value passes a threshold, until it reaches a leaf with the answer. It is learned from examples, choosing at each step the question that best separates the classes. Learned decision trees
deep learning (deep learning)
Machine learning with neural networks of many layers. Each layer transforms what it receives from the one before, and the network learns on its own which features of the data matter.. neural network, machine learning
diffusion model (diffusion model)
A generative model that learns to remove noise. It is trained by corrupting images with noise step by step, and to create a new one it starts from pure noise and cleans it up gradually. The first diffusion model
distillation (distillation)
Training a model, usually a smaller one, on the answers of a more capable one, so that it learns to imitate it at lower cost.. language model
evaluation function (evaluation function)
A formula that scores a game position, for example how much of an advantage White has, without playing the game out to the end. Deep Blue beats Kasparov
expert system (expert system)
A program that solves problems in a specific field, such as medical diagnosis, by applying "if this, then that" rules gathered from human specialists.
feedback (feedback)
A loop in which a system's output is measured and fed back to its input. In negative feedback, the gap between what is measured and the goal corrects the action, as a thermostat does, or a hand reaching for a glass. Behavior, Purpose and Teleology
few-shot learning (few-shot learning)
A model's ability to perform a new task from a few examples included in the request itself, without changing its weights. GPT-3. language model, pretraining
floating point (floating point)
A way of storing a number as a set of significant digits and an exponent that says where the decimal point goes. It makes it possible to represent very large and very small numbers with few digits. Z3
formal system (formal system)
A language of symbols with some axioms and exact rules for deriving new formulas from them. In a formal system, a proof can be checked mechanically, without appeal to intuition. Principia Mathematica. algorithm
foundation model (foundation model)
A large model trained on large amounts of general data that is later adapted to very different tasks. The name was proposed in 2021 by a Stanford University report. The term "foundation model". pretraining, language model
frame problem (frame problem)
The difficulty of describing in logic, without listing everything, what does not change when an action is taken. McCarthy and Hayes posed it in 1969. The frame problem
fuzzy logic (fuzzy logic)
A logic in which a statement such as "the train is going fast" is not simply true or false but holds to a degree between 0 and 1. Fuzzy control on the Sendai subway
GDT (global distance test)
Global distance test: a score from 0 to 100 that measures how closely the predicted structure of a protein matches the real one, by the percentage of amino acids that land near their true position. It is the main measure in the CASP competition. AlphaFold 1
generative adversarial network (generative adversarial network (GAN))
A pair of networks trained in competition. A generator produces samples and a discriminator tries to tell them apart from real ones; in the end, the generator produces data that are hard to distinguish from the real thing. Generative adversarial networks. diffusion model
GPU (graphics processing unit)
Graphics processing unit: a processor designed to draw graphics, which performs many simple operations at once. That ability made it the standard machine for training neural networks.
gradient descent (gradient descent)
A way of reducing a model's error step by step. At each step, every parameter moves a little in the direction in which the error falls fastest.. backpropagation
hidden layer (hidden layer)
A layer of a neural network that neither receives the data nor gives the answer but sits between the two. It is where the network builds its own intermediate representations. Backpropagation by Rumelhart, Hinton, and Williams. neural network, backpropagation
hidden Markov model (hidden Markov model)
A model of a sequence of hidden states, each of which produces observations that can be seen, such as the sounds of a word and the audio signal they produce. Hidden Markov models in speech
interpretability (interpretability)
The study of what happens inside a neural network, what its activations represent, and how it arrives at an answer, so that it is possible to check why it does what it does. Interpretable features in Claude 3 Sonnet. neural network, alignment
kernel trick (kernel trick)
A technique that lets an algorithm work as if the data lived in a space of many more dimensions, where they are easier to separate, without ever computing that projection. All it takes is a function, the kernel, that directly gives the dot product in that space. Support vector machines. support-vector machine
lambda calculus (lambda calculus)
Alonzo Church's formal system in which everything, even numbers, is built by defining and applying functions. It is one of the equivalent definitions of what is computable and the forerunner of functional programming languages. The lambda calculus and an unsolvable problem. algorithm, Turing machine
language model (language model)
A model that assigns probabilities to sequences of words. Today's are trained to predict the next piece of a text, and with that they can write, translate, or answer questions.. token, pretraining
latent space (latent space)
A compressed representation of the data that a model learns. An image is reduced to a much smaller grid of numbers, and the model works on that instead of on the pixels. Stable Diffusion. diffusion model
law of effect (law of effect)
The principle, formulated by Thorndike, that behaviors followed by a satisfying consequence become more likely in the same situation. It is the root of reinforcement learning. The law of effect. reinforcement learning, conditioned reflex
least squares (least squares)
A method for fitting a model to measurements with errors by choosing the parameters that minimize the sum of the squared differences between what was measured and what was predicted. It is the basis of linear regression. The method of least squares. likelihood, gradient descent
likelihood (likelihood)
The probability of the observed data under a given value of a model's parameters. Maximum likelihood estimation means choosing the parameters that make those data most probable. Maximum likelihood. least squares, Bayes' theorem
LSTM (long short-term memory)
Long short-term memory: a type of recurrent network with a memory cell and gates that learn what information goes in, what is kept, and what comes out. It remembers over hundreds or thousands of steps, something simple recurrent networks could not manage. LSTM. recurrent network, vanishing gradient
machine learning (machine learning)
The branch of artificial intelligence in which a program gets better at a task from data or experience, instead of following rules written by hand for each case.. deep learning, reinforcement learning
Markov chain (Markov chain)
A sequence of states in which the probability of moving to the next one depends only on the current state, not on how it was reached. It underlies the first statistical models of language. Markov chains. hidden Markov model, Markov decision process
Markov decision process (Markov decision process)
A mathematical model of a decision problem in which, at each step, what happens next depends only on the current situation and the action chosen. It is the framework of reinforcement learning.. reinforcement learning
minimax (minimax)
A way of choosing a move in a two-player game. Positions at the end of the search are scored, and moving back up the tree, the maximum is taken on one's own turns and the minimum on the opponent's, on the assumption that the opponent also plays as well as possible. Programming a Computer for Playing Chess. evaluation function, alpha-beta pruning
mixture of experts (mixture of experts)
An architecture in which the model has many specialized blocks, the experts, and a router that picks a few of them for each token. The model has many parameters, but each step uses only some of them, which makes it cheaper. Mixtral 8x7B. parameter, token
Monte Carlo tree search (Monte Carlo tree search)
A method for choosing a move that explores the tree of possible games by sampling, playing many simulated games and devoting more effort to the branches that look best. Monte Carlo tree search. evaluation function
multimodal (multimodal)
Said of a model that works with several kinds of data, such as text, images, audio, or video, rather than text alone.. language model
neural network (neural network)
A computing model made of many simple, interconnected units called artificial neurons. Each connection has a weight, and the network learns by adjusting those weights from examples. The McCulloch-Pitts neuron. perceptron, backpropagation, deep learning
neuron doctrine (neuron doctrine)
The theory that the nervous system is made of independent cells, neurons, that communicate by contact rather than forming a continuous network. Ramón y Cajal defended it against Golgi's reticular theory. The neuron doctrine. synapse
open weights (open weights)
Said of a model whose trained parameters are published so anyone can download, run, and adapt it. It does not mean the data or training code are published, and the license may impose conditions. LLaMA. parameter
overfitting (overfitting)
The flaw of a model that fits its training data so closely, noise included, that it gets those examples right but fails on new cases.. machine learning
parameter (parameter)
Each of the numbers a model adjusts from the data. In a neural network they are the weights of its connections, and a language model's size is usually given in billions of parameters.. neural network, pretraining
perceptron (perceptron)
A neural network with a single layer of adjustable weights. It adds up its inputs, each multiplied by its weight, and answers yes or no depending on whether the sum passes a threshold. The perceptron. neural network
Preparedness Framework (Preparedness Framework)
The document in which OpenAI sets out how it measures its models' severe risks, in areas such as biology and cybersecurity, and what safeguards it requires at each level of risk before releasing them.. alignment
pretraining (pre-training)
The first, long phase of training a model, on large amounts of general data. Afterward, a short round of fine-tuning is enough to adapt it to a specific task.. language model
prompt injection (prompt injection)
An attack in which text the model reads while working, such as a web page or an email, contains hidden instructions that the model obeys as if they came from its user.. agent
Q-learning (Q-learning)
A reinforcement learning algorithm that learns, for every situation and every action, how much future reward to expect. It needs no model of the environment, and it learns the best way to act even if the agent does not always act that way while exploring. Q-learning. reinforcement learning, temporal-difference learning
reasoning model (reasoning model)
A language model trained with reinforcement learning to write a long chain of thought before answering. The more computation it spends thinking, the better it usually does on math and programming problems. OpenAI o1. chain of thought, reinforcement learning
recurrent network (recurrent neural network)
A neural network in which signals travel in a loop, so it keeps a state from one step to the next. It is used to process sequences such as sentences or sound. LSTM. neural network
reinforcement learning (reinforcement learning)
Learning in which an agent tries actions in an environment and receives rewards or penalties. It learns what to do in each situation to collect as much reward as possible.. machine learning
relay (relay)
A switch operated by electricity: when current reaches it, it opens or closes another circuit. It has two states, which is why it could be used to build the operations of Boolean algebra out of electricity. Shannon's thesis on relay circuits. Boolean algebra
residual connection (residual connection)
A shortcut that adds the input of a block of layers to its output, so the layers only have to learn a correction. It makes it possible to train networks with many layers. ResNet. deep learning
reward model (reward model)
A network that learns to score behavior from human judgments, such as which of two answers a person prefers. It replaces a hand-written reward in reinforcement learning. Reinforcement learning from human preferences. reinforcement learning, alignment
RLHF (reinforcement learning from human feedback)
Reinforcement learning from human feedback. People rank a model's answers from best to worst, those preferences are used to train a reward model, and the language model learns through reinforcement to score well on it. InstructGPT. reward model, reinforcement learning, alignment
sampling theorem (sampling theorem)
The result that a signal with no frequencies above a given value can be fully reconstructed from samples taken at twice that frequency. It is the basis for converting between continuous signals and digital data. Kotelnikov's sampling theorem
scaling laws (scaling laws)
Measured relationships that predict how a model's error falls as its parameters, data, or training compute increase. They follow power laws, with diminishing returns. Scaling laws. language model
support-vector machine (support-vector machine)
A classifier that separates two classes with the boundary that leaves the widest possible margin to the nearest examples on each side, called support vectors. With the kernel trick, it can also separate data that admit no straight boundary. Support vector machines. kernel trick
synapse (synapse)
The point of contact through which one neuron passes a signal to another. Its strength changes with experience, and in an artificial neural network its counterpart is the weight of each connection. The word "synapse". neural network, neuron doctrine
temporal-difference learning (temporal-difference learning)
A learning method that corrects each prediction with the next prediction, without waiting for the final outcome. Temporal differences. reinforcement learning
token (token)
A piece of text, as split up before it is fed to a language model. It can be a word, part of a word, or a punctuation mark.. language model
Transformer (Transformer)
A neural network architecture built entirely from attention mechanisms, with no recurrence, that processes all the words in a sequence at once. It is the basis of large language models. The Transformer. attention mechanism, language model
Turing machine (Turing machine)
A mathematical model of computation with a tape of cells, a head that reads and writes symbols, and a finite table of rules. Anything a computer can compute, a Turing machine can in principle compute too. On Computable Numbers. universal machine, algorithm
Turing test (Turing test)
A test Turing proposed in 1950. An interrogator converses in writing with a hidden person and a hidden machine and must work out which is which; if the machine fools the interrogator as often as a person would, it is credited with intelligent behavior. Computing Machinery and Intelligence
universal machine (universal machine)
A Turing machine that takes the description of another machine on its tape and imitates it step by step. It is the theoretical idea behind the stored-program computer, in which program and data are kept together. On Computable Numbers. Turing machine
unsupervised learning (unsupervised learning)
Learning from unlabeled data: nobody tells the model what the right answer is, and it has to find the structure of the data on its own. Kohonen's self-organizing maps. machine learning
vanishing gradient (vanishing gradient)
A problem in training deep or recurrent networks. As the error is backpropagated through many layers or steps, the gradient is multiplied again and again by small numbers and approaches zero, so the first layers barely learn. If the numbers are large, it blows up instead. LSTM. backpropagation, LSTM
VC dimension (VC dimension)
A measure of how flexible a family of classifiers is, from Vapnik and Chervonenkis. It is the largest number of points the family can split into two classes in every possible way; straight lines in the plane have dimension 3. Vapnik-Chervonenkis theory. overfitting
vision-language-action model (vision-language-action model)
A model that sees through a camera, understands instructions in natural language, and responds with the movement commands for a robot. Gemini Robotics. multimodal
word embedding (word embedding)
A list of numbers that represents a word as a point in a space of many dimensions. Words that appear in similar contexts end up close together. The neural probabilistic language model. language model
world model (world model)
A model that learns how an environment changes and can simulate what will happen next, for example by generating a video world that responds to the actions of whoever moves through it. Genie 3