The History of AI

IBM's statistical machine translation

Era 3 · From the expert boom to the statistical turn (1980–2011) › Statistical learning and probabilistic models

Lineage
Data and learning
Kind
Publication

Who

IBM

What happened

In 1988, at the COLING conference in Budapest, Peter Brown, Robert Mercer, Frederick Jelinek, and their colleagues at IBM's research center present a statistically based machine translation. They describe it in 1990 and 1993 in Computational Linguistics.

Why it mattered

Data replace linguistic rules: the system learns to translate from texts that have already been translated. It is the approach that dominated machine translation until neural networks arrived.

The full story

Since the ALPAC report, the machine translation that survived had been based on rules written by linguists: dictionaries, grammars, and transformations from one language to another. The IBM group came from speech recognition, where it worked with statistical models learned from data.

Its idea was to treat translation as a decoding problem. To translate from French into English, you look for the English sentence most likely to have produced the French one. That probability breaks down into two parts: a language model, which measures whether the sentence is good English, and a translation model, which measures whether the words correspond. Both are estimated from data. For the second, they used the bilingual proceedings of the Canadian Parliament, millions of sentences in English and French, and learned automatically which words align with which.

The 1993 paper presented a series of five models of increasing complexity, known as IBM Models 1 to 5, which became the foundation of the discipline.

Statistical translation dominated until the mid-2010s, when neural sequence-to-sequence models with attention displaced it.

Three boxes in a row: English sentence, channel, and French sentence, joined by arrows. A return arrow runs from the French sentence to the English one, labeled translating means retracing the path. Below, two boxes multiplied together: the language model, which measures whether it is good English, and the translation model, which measures whether the words match.

IBM treated French as English distorted by a channel. Translating means finding the most probable English sentence, combining a language model and a translation model.

Builds on

  • The ALPAC report (November 1966)

    Inherited, with no proven direct link

    After the ALPAC report, the machine translation that survived was rule-based; the IBM group replaces it with a model learned from bilingual text

Leads to

  • Sequence to sequence (June 3, 2014)

    Inherited, with no proven direct link

    Sequence-to-sequence neural translation is measured against phrase-based statistical translation, the system it comes to replace

Sources

Original source

How to cite

“IBM's statistical machine translation.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/ibm-statistical-translation

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline