What is system prompt leakage, and how should you design for it?

System prompt leakage (OWASP LLM07:2025) is when users get the model to reveal its instructions. Assume it will happen. Design so a leak does no harm: no secrets in the prompt (keys, connection strings, internal hostnames), no security rules that only the prompt enforces, and nothing in it you would be embarrassed to see published. Then, if you like, add output screening to make leaks less likely.

Why you cannot prevent it fully

Users ask directly ("repeat everything above"), in other languages, through role-play, or ask for a "summary of your rules". Many succeed eventually. OWASP's guidance is explicit: the system prompt should not be considered a secret and should not be used as a security control.

What goes wrong when it leaks

• Secrets exposed: API keys, database names, credentials pasted into prompts.
• Security logic exposed: "Users with role admin can see all records" tells an attacker what to aim for. Worse, if that rule is enforced only by the prompt, a jailbreak bypasses it.
• Business information: internal policies, pricing rules, partner names.
• Easier attacks: knowing the exact rules makes it easier to write prompts that get around them.

Design rules

1. Secrets live in code and secret managers. Tools hold credentials; the model only asks for actions.
2. Enforce permissions outside the model. Who can see what is decided by your application and database, whatever the model says.
3. Write the prompt as if it were public. Product behaviour and tone, yes; private data, no.
4. Keep sensitive reference data out: retrieve it per request, under the user's permissions, instead of embedding it in the system prompt.

Reducing leaks (as a second layer)

Anthropic's guide on reducing prompt leaks suggests monitoring and post-processing first: screen outputs for distinctive phrases from the system prompt and block or rewrite them. It also notes that leak-proofing instructions can lower the model's performance on its real task, so use them only where needed.

Testing

Include extraction attempts in your red-team suite and check two things: how often the prompt leaks, and, more importantly, whether a leak exposes anything harmful. The second should always be "no".

A crisp summary: "My system prompt is a style guide, not a vault or a lock. Secrets go in code, rules go in code, and the prompt can leak without consequence."