A support agent issued a refund it was not allowed to give. How do you fix this so it cannot happen again?

Move the rule out of the prompt and into the code that performs the refund. A prompt that says "never refund more than $100" is a request the model usually follows, not a control. Users can argue with it, injected text can override it, and a model can simply get it wrong. The refund tool must check the policy itself, using data the model cannot change, and refuse or escalate anything outside it.

1. Contain and understand

• Find every refund the agent issued, not just the one reported, and check each against the policy.
• Read the trace of the bad run: the conversation, the tool calls and their arguments. Did a user persuade it ("my manager approved this"), did text in a document or email instruct it (prompt injection), or did it misread the policy?
• If needed, switch the tool to "propose only" until the fix is live.

2. Enforce the policy in the tool

The refund tool should take only what it needs from the model, and look up everything else itself:

def issue_refund(order_id: str, amount: Decimal, reason: str, ctx: Session):
    order = orders.get(order_id)
    if order.customer_id != ctx.customer_id:           # from the login, not the model
        return error("This order belongs to a different customer.")
    if amount > order.paid - order.refunded:
        return error("Refund exceeds the amount left on the order.")
    if amount > policy.auto_refund_limit(order):
        return escalate(order, amount, reason)         # a person decides
    return payments.refund(order, amount, idempotency_key=f"refund-{order_id}-{amount}")

The customer's identity comes from the authenticated session, never from what the model passes in. The limit comes from the policy service. The model can ask for anything; the code only does what is allowed.

3. Limit what the agent can do at all

OWASP's guidance on excessive agency names three causes: too much functionality, too many permissions and too much autonomy. Apply each:

• Functionality: give the agent issue_refund, not a general call_payments_api.
• Permissions: the service account behind the tool can refund, but cannot change prices or customer records.
• Autonomy: above a threshold, or when the request looks unusual, a person approves before the action happens.

4. Make refusals useful

When the tool refuses, return a clear message the agent can act on ("Above the automatic limit; a request has been sent to a supervisor, tell the customer they will hear back within one working day"). The agent then explains the outcome instead of retrying or inventing one.

5. Prove it holds

Add the incident to the evaluation set, with variations: users who claim approval, split requests that add up past the limit, instructions hidden in an uploaded document. Run them on every change to the prompt, model or tools. Monitor refunds per conversation and per customer, and alert on anything unusual.

What interviewers want: the clear statement that a prompt is not an access control, authorization enforced in the tool with identity from the session, least privilege and human approval for high-impact actions, and tests built from the incident.