The History of AI

Variational autoencoder

Era 4 · Deep learning and foundation models (2012–2021) › Generative image models

Lineage
Neurons and networks
Kind
Publication

Who

University of Amsterdam

What happened

Diederik Kingma and Max Welling of the University of Amsterdam post "Auto-Encoding Variational Bayes" on arXiv, a method for training models with continuous latent variables by gradient descent. The resulting model is known as the variational autoencoder.

Why it mattered

A model that learns to compress each image into a few variables with a probabilistic meaning, and to generate new images from them.

The full story

An autoencoder is a network that learns to compress a piece of data, such as an image, into a small vector and then reconstruct it. It is good for summarizing data but not for creating new data: there is no way to know which vectors produce meaningful images.

Kingma and Welling approached the problem from Bayesian statistics. They assume that each image comes from a few hidden variables, and they want to learn at the same time how the image is generated from those variables and how to infer the variables from the image. That second calculation was intractable, so they train a recognition network that approximates it.

A month later Danilo Rezende, Shakir Mohamed, and Daan Wierstra of DeepMind in London independently published an equivalent method. The shared technical contribution was a reparameterization: instead of sampling the hidden variables directly, they sample a fixed noise and transform it with the network's parameters. With that trick, the bound to be optimized can be differentiated and trained with ordinary stochastic gradient descent, and the method scales to large datasets.

The result is an orderly space of hidden variables: nearby points give similar images, and sampling a point is all it takes to generate a new image. The idea of working in a compressed latent space became a component of many later generators: Stable Diffusion, for example, includes a variational autoencoder and generates its images in that reduced space rather than pixel by pixel.

Leads to

  • DALL·E and CLIP (January 5, 2021)

    Documented direct influence

    DALL·E turns each image into tokens with a discrete variational autoencoder (dVAE), its first training stage (section 2)

  • Stable Diffusion (August 22, 2022)

    Documented direct influence

    Latent diffusion works in the space of an autoencoder; one of its variants imposes a slight KL penalty, "similar to a VAE," citing Kingma and Welling

Threads that pass through here

Sources

How to cite

“Variational autoencoder.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/variational-autoencoder

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline