The full answer
guardrails are the checks around a model that keep inputs and outputs inside what the product allows: what may go in, what may come out, and what actions may happen. Put them in layers: before the model (input checks), in the request (instructions, schemas, tool permissions), after the model (output checks), and around actions (permissions and approvals). Then monitor what they catch. No single layer is reliable alone.
1. Before the model: input guardrails
- Scope: is the request something this product handles? A cheap classifier routes off-topic requests away.
- Abuse and injection screening: Anthropic suggests a harmlessness screen, a small fast model that flags harmful or manipulative input before the main call.
- Sensitive data: detect and mask personal data that should not reach the model or logs.
- Size limits: cap input length and request rate.
2. In the request
- A clear system prompt with what the assistant does and does not do. This helps, but treat it as guidance, not enforcement.
- Structured outputs so responses fit a schema.
- Tool permissions: only the tools the task needs, with argument allow-lists.
3. After the model: output guardrails
- Policy classifier: does the answer break the content policy? (Anthropic's content moderation guide shows a model classifying text against a written policy.)
- Grounding check: are claims supported by the sources?
- Format and value validation in code: schema, ranges, ids that must exist.
- Leak checks: no system prompt text, keys or other users' data.
- Safe rendering: escape before HTML, never execute output directly.
4. Around actions
Irreversible or external actions (send, pay, delete) need a permission check in code and, often, a person's approval. This is the layer that limits damage when everything above fails.
5. Monitor
Log what each guardrail blocked and why, review samples, and check false positives. A guardrail that blocks 5% of legitimate requests is a product bug.
Trade-offs to mention
- Each check adds latency and cost. Run cheap checks in parallel with the main call where possible, and stream only after fast output checks, or check in chunks.
- Classifiers are models too: they miss things and can be fooled. Deterministic code checks are the strongest layer for anything that can be expressed as a rule.
- Too strict, and the product is useless. Tune thresholds on a labelled set of both allowed and disallowed requests.
A one-line summary: "Prompts ask, guardrails check, and permissions enforce. I rely on the last two for anything that matters."
Last updated 16 October 2026
Sources