The History of AI

Bias and accountability in AI

Era 4 · Deep learning and foundation models (2012–2021) › Alignment

Lineage
Data and learning
Kind
Publication

Who

MIT Media Lab, Microsoft Research, University of Washington, Google

What happened

In February 2018, in "Gender Shades," Joy Buolamwini and Timnit Gebru show that three commercial facial analysis systems fail far more often with darker-skinned women. In March 2021 Emily M. Bender, Gebru, and other coauthors warn of the risks of ever larger language models.

Why it mattered

Alongside the question of how to make a system do what is wanted, another one takes hold: whom it harms when it fails, and who answers for it.

The full story

Machine learning systems learn from the data they are trained on, and they inherit its gaps. Buolamwini, at the MIT Media Lab, and Gebru, at Microsoft Research, measured this in facial analysis. Two widely used benchmark datasets, IJB-A and Adience, were made up of 79.6 percent and 86.2 percent lighter-skinned people. Using a new dataset balanced by sex and skin type, they evaluated three commercial gender classification systems.

The result, presented at the FAT* conference in February 2018, was that darker-skinned women were the worst-classified group, with error rates of up to 34.7 percent, against a maximum of 0.8 percent for lighter-skinned men.

In 2020, by then at Google, Gebru wrote "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" with Emily M. Bender of the University of Washington and other coauthors. The paper pointed to the financial and environmental cost of training such models, to the hard-to-detect biases they absorb from huge amounts of text, and to the risk that they mislead, because they produce fluent language with no grounding in meaning.

In December 2020 Gebru left Google after the company asked for the paper to be withdrawn; Google said it was accepting Gebru's resignation, and Gebru said no resignation had been offered. The paper was published in March 2021 at the FAccT conference.

Sources

How to cite

“Bias and accountability in AI.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/bias-and-accountability

This entry was last reviewed on November 13, 2026.

Open in the interactive timeline