The History of AI

DeepSeek-V3

Era 5 · Generative AI, reasoning, and agents (2022–2026) › Frontier models

Lineage
Neurons and networks
Kind
Implementation

Who

DeepSeek

What happened

DeepSeek releases DeepSeek-V3, a 671-billion-parameter mixture-of-experts model with open weights. Its technical report says the full training run took 2.788 million H800 GPU hours, about $5.6 million at rental prices.

Why it mattered

An open model comparable to the best closed ones, trained with far less compute than usual and on chips cut down by export controls. It is the basis of DeepSeek-R1.

The full story

In late 2024 US export controls prevented Chinese labs from buying the most advanced GPUs. DeepSeek trained on the H800, a version Nvidia had cut down so that it could be sold in China.

On December 26 it presented DeepSeek-V3 and released the weights. It is a mixture-of-experts model with 671 billion parameters, of which it uses 37 billion per token. It was trained on 14.8 trillion tokens of text on a cluster of 2,048 H800 GPUs.

Its technical report, published the next day, credited the efficiency to several techniques: an attention mechanism that compresses the memory the model keeps of what it has already read, a way of balancing the load across experts that does not penalize learning, and training to predict several tokens at once. The full training run had consumed 2.788 million GPU hours, about $5.6 million at $2 an hour. The authors cautioned that the figure excluded prior research and experiments, but it was the one that circulated.

According to the report, V3 outperformed the other open models and came close to the best closed ones, such as GPT-4o and Claude 3.5 Sonnet.

Three and a half weeks later DeepSeek released R1, a reasoning model built on V3. The combination of the two, and the reported cost, triggered Nvidia's stock market plunge on January 27, 2025.

Builds on

  • Chip export controls on China (October 7, 2022)

    Inherited, with no proven direct link

    DeepSeek-V3 is trained on H800 GPUs, the lower-bandwidth version Nvidia sold in China under the export controls

  • Mixtral 8x7B (December 11, 2023)

    Inherited, with no proven direct link

    Both are open sparse mixture-of-experts models that activate only part of their parameters for each token

Leads to

  • DeepSeek-R1 (January 20, 2025)

    Documented direct influence

    R1-Zero and R1 are trained from DeepSeek-V3-Base (section 2)

Threads that pass through here

Sources

Original source

How to cite

“DeepSeek-V3.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/deepseek-v3

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline