Constitutional AI and Safety
Constitutional AI uses written principles to critique and improve model behavior.
Instead of relying only on humans to label every preference, a model can critique a response against a set of principles and revise it.
The core loop
draft answer -> critique against principles -> revise answer -> preference training
The constitution is not a law book. It is a set of behavior principles, such as being helpful, honest, harmless, privacy-preserving, and non-deceptive.
Why it matters
Production AI systems need to avoid two extremes:
| Bad behavior | Example |
|---|---|
| unsafe compliance | helping with harmful actions |
| over-refusal | refusing harmless educational requests |
Good safety design is calibrated.
Product guardrails
Constitutional AI is one training approach. Applications still need runtime guardrails:
- input classification
- tool permission checks
- output validation
- human approval
- red-team tests
- monitoring and incident review
Principle design
Useful principles are:
- specific enough to guide behavior
- broad enough to generalize
- testable with examples
- reviewed by domain experts
- updated when incidents reveal gaps
Knowledge check
Q1: What is the basic Constitutional AI loop?
Draft, critique against principles, revise, and use preferences for training.
Q2: Why are runtime guardrails still needed?
Training cannot anticipate every product context, tool permission, or attack.