The History of AI

CUDA

Era 3 · From the expert boom to the statistical turn (1980–2011) › Hardware and data

Lineage
Machines and computation
Kind
Implementation

Who

Nvidia

What happened

Nvidia releases the CUDA software development kit, a C programming interface that lets its graphics processing units (GPUs) be used for general computation and not just for drawing images.

Why it mattered

GPUs, with thousands of operations running in parallel, come within reach of any programmer. A few years later they become the engine of neural network training.

The full story

A graphics card computes the color of millions of pixels at once, and each pixel is worked out with the same operations on different data. In the early 2000s some researchers realized that this parallelism could serve other calculations, but to exploit it they had to disguise their problems as graphics operations.

At Stanford University, Ian Buck developed Brook, a language for programming GPUs as general-purpose processors. Nvidia hired Buck in 2004, and Buck worked with John Nickolls on what would become CUDA: an extension of the C language in which the programmer writes a function and the GPU runs it on thousands of threads at once.

Nvidia announced CUDA in November 2006, along with the GeForce 8800, and released the development kit in February 2007. The following year Nickolls, Buck, and others presented it as a parallel programming model that scales automatically to the number of cores on each card, which meant code could be written once and get faster with every new generation of hardware.

CUDA was not designed for artificial intelligence but for science and engineering. Yet training neural networks consists mostly of multiplying matrices, exactly the kind of computation a GPU does best, and over the following decade machine learning became the main reason to buy compute GPUs.

Leads to

  • AlexNet (September 30, 2012)

    Documented direct influence

    AlexNet is trained on GPUs with CUDA, on ImageNet

  • Deep networks trained on GPUs (June 2009)

    Documented direct influence

    The paper programs Nvidia GPUs with CUDA to spread learning across thousands of threads

Threads that pass through here

Sources

Original source

How to cite

“CUDA.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/cuda

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline