Constitutional AI
Era 5 · Generative AI, reasoning, and agents (2022–2026) › Alignment and reasoning
- Lineage
- Data and learning
- Kind
- Publication
Who
Anthropic
What happened
Yuntao Bai and colleagues at Anthropic present constitutional AI, a method for training a harmless assistant without human labels on harmful responses: people only write a list of principles, and the model critiques and revises its own responses based on them.
Why it mattered
The model is trained on a set of written principles and on feedback from another AI, instead of thousands of human judgments. It responds by explaining its objections instead of simply refusing.
The full story
Learning from human feedback required people to read and rate large numbers of responses, harmful ones included, to teach the model what to avoid. It took a great many human labels, and the criteria remained implicit in thousands of individual judgments that nobody had ever written down as rules.
Constitutional AI replaces those judgments with an explicit list of principles, the constitution. In a first, supervised phase, the model generates a response, critiques it against the principles, rewrites it, and is fine-tuned on the revised versions. In a second, reinforcement learning phase, another model compares pairs of responses and decides which one better follows the principles. Those AI preferences train the reward model, and the authors called the whole approach reinforcement learning from AI feedback.
The resulting assistant was harmless without being evasive: faced with a harmful request, it explained its objections instead of merely refusing. The authors noted that asking the model to reason step by step when judging improved the quality and transparency of its decisions, and that the method made it possible to control behavior with far fewer human labels.
Anthropic applied this method to its Claude models, released from March 2023 on.
Builds on
InstructGPT (March 4, 2022)
Inherited, with no proven direct link
Constitutional AI keeps the scheme of reinforcement learning with a preference model and replaces human labels on harmful responses with labels from an AI guided by principles
Sources
Original source
- Constitutional AI: Harmlessness from AI Feedback (arXiv:2212.08073) (opens in a new tab). Yuntao Bai et al., 2022. open license: CC BY 4.0
- Constitutional AI: Harmlessness from AI Feedback (Anthropic, December 15, 2022) (opens in a new tab). Anthropic, 2022
Scholarship and context
How to cite
“Constitutional AI.” The History of AI, revised November 13, 2026. https://thehistoryofai.com/en/milestone/constitutional-ai
This entry was last reviewed on November 13, 2026.