The History of AI

A Mathematical Theory of Communication

Era 2 · AI is born (1943–1979) › Cybernetics, information, and electronic computers

Lineage
Machines and computation
Kind
Publication

Who

Bell Labs

What happened

In two installments of the Bell System Technical Journal, in July and October 1948, Claude Shannon publishes a mathematical theory of how messages are transmitted. In 1949 it comes out as a book with a commentary by Warren Weaver.

Why it mattered

It defines information as something measurable, with the bit as its unit and entropy as the measure of uncertainty, and proves that every channel has a maximum capacity for error-free transmission.

The full story

Telecommunications engineers in the 1940s knew a great deal about signals, noise, and bandwidth, but had no measure of what was being transmitted. Shannon, working at Bell Labs, proposed separating information entirely from meaning: what matters is how many possible messages there are and how likely each one is.

The resulting measure, entropy, is counted in bits, a word Shannon credits to a colleague, John Tukey, and which appears here in print for the first time. A bit is the information provided by the answer to a yes-or-no question with two equally likely options. With this measure, Shannon proved two results that founded the discipline: a message can be compressed down to its entropy and no further, and every noisy channel has a capacity below which information can be sent with an error as small as desired.

The paper also includes an experiment that anticipates today's language models: it generates English text by choosing each letter or word according to the probability that it follows the previous ones, and shows how the approximations come to look more and more like real English.

Information theory became the basis of digital telecommunications, data compression, and cryptography, and its notions of entropy and conditional probability run through machine learning to this day.

Videos

Builds on

  • Markov chains (1906)

    Documented direct influence

    Shannon models the source of a message as a Markov process and illustrates it with letter-by-letter approximations to English

  • Transmission of Information (July 1928)

    Documented direct influence

    The introduction cites Nyquist and Hartley as forerunners and credits Hartley with the logarithmic measure of information

  • Kotelnikov's sampling theorem (1933)

    Independent convergence

    The sampling theorem appears in Shannon's work with no reference to Kotelnikov, whose paper was unknown outside the USSR

Sources

How to cite

“A Mathematical Theory of Communication.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/shannon-information-theory

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline