The History of AI

Vision Transformer

Era 4 · Deep learning and foundation models (2012–2021) › Transformers and language models

Lineage
Neurons and networks
Kind
Publication

Who

Google

What happened

Alexey Dosovitskiy and eleven coauthors post "An Image is Worth 16x16 Words" to arXiv on October 22, 2020. They apply a standard Transformer, with no convolutions, to image classification. The final version is presented at ICLR 2021.

Why it mattered

The architecture built for language works for vision too: pretrained on enough data, the Vision Transformer matches or beats the best convolutional networks while using substantially less training compute.

The full story

Since AlexNet in 2012, computer vision had run on convolutional networks, designed specifically for images. The Transformer had pushed recurrent networks out of language, but in vision, attention was used alongside convolutions or in place of some of their parts, never without them.

The Vision Transformer tries the simplest thing possible. It cuts the image into fixed-size patches (16 by 16 pixels, hence the title), turns each patch into a vector, and hands the sequence to a standard Transformer as if every patch were a word. There is not a single convolution.

Trained only on mid-sized datasets like ImageNet, the model came in a few points below a ResNet convolutional network of similar size. The authors put this down to the Transformer lacking the built-in assumptions about images that convolutions carry. The picture changes with more data. Pretrained on datasets of 14 million to 300 million images (ImageNet-21k and JFT-300M, an internal Google dataset) and then fine-tuned on mid-sized and small benchmarks, the Vision Transformer matched or beat the best convolutional networks and needed substantially less compute to train.

The final version of the paper is the one presented at ICLR 2021, and the code and pretrained models were released on GitHub. With this work, the Transformer became a general-purpose architecture, no longer just a text model.

Builds on

  • The Transformer (June 12, 2017)

    Documented direct influence

    The paper applies "a pure transformer" directly "to sequences of image patches"

Leads to

  • Sora (February 15, 2024)

    Inherited, with no proven direct link

    Sora is a diffusion model that works on a compressed representation of video, split into patches like the images in the Vision Transformer

Threads that pass through here

Sources

Original source

How to cite

“Vision Transformer.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/vision-transformer

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline