A Mathematical Theory of Communication
Era 2 · AI is born (1943–1979) › Cybernetics, information, and electronic computers
- Lineage
- Machines and computation
- Kind
- Publication
Who
Bell Labs
What happened
In two installments of the Bell System Technical Journal, in July and October 1948, Claude Shannon publishes a mathematical theory of how messages are transmitted. In 1949 it comes out as a book with a commentary by Warren Weaver.
Why it mattered
It defines information as something measurable, with the bit as its unit and entropy as the measure of uncertainty, and proves that every channel has a maximum capacity for error-free transmission.
The full story
Telecommunications engineers in the 1940s knew a great deal about signals, noise, and bandwidth, but had no measure of what was being transmitted. Shannon, working at Bell Labs, proposed separating information entirely from meaning: what matters is how many possible messages there are and how likely each one is.
The resulting measure, entropy, is counted in bits, a word Shannon credits to a colleague, John Tukey, and which appears here in print for the first time. A bit is the information provided by the answer to a yes-or-no question with two equally likely options. With this measure, Shannon proved two results that founded the discipline: a message can be compressed down to its entropy and no further, and every noisy channel has a capacity below which information can be sent with an error as small as desired.
The paper also includes an experiment that anticipates today's language models: it generates English text by choosing each letter or word according to the probability that it follows the previous ones, and shows how the approximations come to look more and more like real English.
Information theory became the basis of digital telecommunications, data compression, and cryptography, and its notions of entropy and conditional probability run through machine learning to this day.
Videos
- Claude Shannon - Father of the Information Age (opens in a new tab) (University of California Television (UCTV))
Builds on
Markov chains (1906)
Documented direct influence
Shannon models the source of a message as a Markov process and illustrates it with letter-by-letter approximations to English
Transmission of Information (July 1928)
Documented direct influence
The introduction cites Nyquist and Hartley as forerunners and credits Hartley with the logarithmic measure of information
Kotelnikov's sampling theorem (1933)
Independent convergence
The sampling theorem appears in Shannon's work with no reference to Kotelnikov, whose paper was unknown outside the USSR
Sources
Original source
How to cite
“A Mathematical Theory of Communication.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/shannon-information-theory
This entry was last reviewed on November 13, 2026.