Data, GPUs, and AlexNet
Programmable GPUs and a massive set of labeled images, the two missing ingredients, come together in AlexNet.
CUDA (February 15, 2007)
Nvidia releases CUDA: GPUs can now be programmed for general-purpose computing.
ImageNet (June 2009)
Meanwhile, Fei-Fei Li and collaborators gather millions of hand-labeled images in ImageNet; the 2012 competition uses 1.2 million in a thousand categories. It is AlexNet's other ingredient, and it has nothing to do with GPUs.
DanNet (2011)
In 2011, DanNet, a convolutional network trained on GPUs at IDSIA, the Swiss lab, beats humans in a traffic sign recognition competition.
AlexNet (September 30, 2012)
AlexNet is trained on two GPUs with CUDA on ImageNet, with an architecture its paper compares to DanNet's. Its top-5 error is 15.3 percent, more than 10 points below the runner-up.