From diffusion to world models
Diffusion models go from generating images to generating video and, later, interactive worlds that respond to whoever moves through them.
DDPM (June 19, 2020)
Diffusion models learn to remove noise step by step until an image emerges.
Vision Transformer (October 22, 2020)
Meanwhile, the Vision Transformer treats an image as a sequence of patches.
DALL·E and CLIP (January 5, 2021)
Meanwhile, OpenAI introduces CLIP, which places images and text in the same space.
DALL·E 2 (April 6, 2022)
DALL·E 2 starts from CLIP's representation and generates the image with a diffusion decoder.
Stable Diffusion (August 22, 2022)
Stable Diffusion moves diffusion into a compressed space and releases its weights. It runs on a home computer.
Sora (February 15, 2024)
Sora applies diffusion to video, on a compressed representation cut into patches, and generates clips up to a minute long.
Genie 3 (August 5, 2025)
Genie 3 does not generate a finished video but a world you can move through in real time.