A harmful or wrong AI output reached the customer's end users. How do you respond?

Handle it as an incident: contain it first (switch off or restrict the affected path), assess the impact (who saw what, what harm, what obligations), inform the customer quickly and honestly, find the root cause from traces, fix and add tests so it cannot recur, and review what controls failed. Speed and honesty matter more than a perfect first explanation.

1. Contain (minutes to hours)

• Use the kill switch or feature flag: turn off automatic sending, fall back to human review, or block the affected category.
• Stop further damage before investigating in depth.
• Preserve the evidence: traces, prompts, retrieved documents, model and prompt versions.

2. Assess impact

• How many users received similar outputs, and when? Search logs for the pattern.
• What kind of harm: wrong information, offensive content, personal data exposed, an action taken?
• Are there legal or contractual duties: data breach notification, regulator reporting, contract terms on incidents? Involve the customer's legal and security teams early.

3. Communicate

• Tell the customer's sponsor and incident contact quickly: what happened, what you have done, what you are still finding out, when the next update comes.
• Help them prepare messages to affected users, and corrections if wrong information was given.
• Never minimise or guess. "We don't know yet; next update at 15:00" is fine.

4. Root cause

From the traces, find the chain: what input, what retrieval, what prompt, what output, and why no control caught it. Typical causes:

• a new kind of input the system was never tested on,
• a document with wrong or injected content,
• a model or prompt change that skipped the evaluation gate,
• an output check that was missing or too loose.

5. Fix and prevent

• Fix the immediate cause.
• Add the case, and variations of it, to the evaluation and red-team sets.
• Strengthen the control that should have caught it: an output check, human review for that category, a permission.
• Re-enable gradually, with closer monitoring.

6. Review

A blameless written review with the customer: timeline, impact, causes, fixes, and follow-up actions with owners and dates.

What makes this easier

Systems built with kill switches, full traces, versioned prompts and models, and an incident plan agreed before launch. Building those in is part of the deployment, not an extra.

What interviewers look for: containment first, honest communication, data-driven root cause, and fixes that change the system, not only the prompt.