IBM's statistical machine translation
Era 3 · From the expert boom to the statistical turn (1980–2011) › Statistical learning and probabilistic models
- Lineage
- Data and learning
- Kind
- Publication
Who
IBM
What happened
In 1988, at the COLING conference in Budapest, Peter Brown, Robert Mercer, Frederick Jelinek, and their colleagues at IBM's research center present a statistically based machine translation. They describe it in 1990 and 1993 in Computational Linguistics.
Why it mattered
Data replace linguistic rules: the system learns to translate from texts that have already been translated. It is the approach that dominated machine translation until neural networks arrived.
The full story
Since the ALPAC report, the machine translation that survived had been based on rules written by linguists: dictionaries, grammars, and transformations from one language to another. The IBM group came from speech recognition, where it worked with statistical models learned from data.
Its idea was to treat translation as a decoding problem. To translate from French into English, you look for the English sentence most likely to have produced the French one. That probability breaks down into two parts: a language model, which measures whether the sentence is good English, and a translation model, which measures whether the words correspond. Both are estimated from data. For the second, they used the bilingual proceedings of the Canadian Parliament, millions of sentences in English and French, and learned automatically which words align with which.
The 1993 paper presented a series of five models of increasing complexity, known as IBM Models 1 to 5, which became the foundation of the discipline.
Statistical translation dominated until the mid-2010s, when neural sequence-to-sequence models with attention displaced it.
Three boxes in a row: English sentence, channel, and French sentence, joined by arrows. A return arrow runs from the French sentence to the English one, labeled translating means retracing the path. Below, two boxes multiplied together: the language model, which measures whether it is good English, and the translation model, which measures whether the words match.
Builds on
The ALPAC report (November 1966)
Inherited, with no proven direct link
After the ALPAC report, the machine translation that survived was rule-based; the IBM group replaces it with a model learned from bilingual text
Leads to
Sequence to sequence (June 3, 2014)
Inherited, with no proven direct link
Sequence-to-sequence neural translation is measured against phrase-based statistical translation, the system it comes to replace
Sources
Original source
- A Statistical Approach to Language Translation. Coling Budapest 1988, vol. 1 (ACL Anthology) (opens in a new tab). Peter Brown, John Cocke, Stephen Della Pietra, Vincent Della Pietra, Frederick Jelinek, Robert Mercer, and Paul Roossin, 1988
- A Statistical Approach to Machine Translation. Computational Linguistics, 16(2), pp. 79-85 (ACL Anthology) (opens in a new tab). Peter F. Brown, John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer, and Paul S. Roossin, 1990
- The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, 19(2), pp. 263-311 (ACL Anthology) (opens in a new tab). Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer, 1993
How to cite
“IBM's statistical machine translation.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/ibm-statistical-translation
This entry was last reviewed on November 13, 2026.