The History of AI

Deep networks in speech recognition

Era 3 · From the expert boom to the statistical turn (1980–2011) › Neural networks

Lineage
Neurons and networks
Kind
Application

Who

Microsoft Research

What happened

Frank Seide, Gang Li, and Dong Yu of Microsoft Research present a deep network at Interspeech 2011 that transcribes telephone conversations with a third fewer errors than the best earlier systems.

Why it mattered

Deep learning's first major industrial success, a year before AlexNet. Within a few years every major speech recognizer switches to deep networks.

The full story

For two decades, speech recognizers combined hidden Markov models, which track the sequence of sounds, with Gaussian mixture models, which estimate which sound matches each slice of audio. Improvements came year after year in small steps.

In 2009 Hinton's students in Toronto showed that a deep network, pretrained layer by layer like the 2006 deep belief nets, recognized phonemes better than those mixtures. Li Deng and Dong Yu of Microsoft Research took an interest in the result, and together with George Dahl, one of Hinton's students, they carried the idea over to large-vocabulary recognition.

The system built by Seide, Li, and Yu replaced the Gaussian mixtures with a network of up to nine hidden layers trained on more than 300 hours of conversations. On the standard telephone conversation benchmark, the word error rate fell from 27.4 percent to 18.5 percent: a third less in a single stroke, in a field that had spent years advancing a point at a time.

In 2012 the Toronto, Microsoft, Google, and IBM groups published a joint overview that confirmed the result on their own systems. In the years that followed, deep networks went into commercial recognizers, and speech became the first area where deep learning changed a mass-market product.

Builds on

  • Hidden Markov models in speech (February 1989)

    Documented direct influence

    The deep network replaces the Gaussian mixtures inside a recognizer built on hidden Markov models

  • Deep belief nets (July 2006)

    Documented direct influence

    The recognizer's networks are pretrained layer by layer as deep belief networks

Sources

How to cite

“Deep networks in speech recognition.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/deep-networks-speech-recognition

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline