A customer says "we want an AI agent for customer support." How do you scope the first pilot?

Turn the wish into one narrow, measurable workflow. Find out what support work costs them most, pick a slice where AI can help with low risk, agree a success metric with a baseline and a test set from their own tickets, define what the AI may and may not do, and time-box the pilot (often 4 to 8 weeks) with a clear decision at the end: expand, change course or stop.

1. Discover before proposing

• Who are the users: customers, support agents, or both?
• Volumes, ticket categories, current handle time, resolution rate, cost per ticket, satisfaction scores.
• Which systems hold the answers: help centre, CRM, order system, policies?
• What goes wrong today, and what does leadership actually want: lower cost, faster replies, 24-hour cover, better quality?
• Sit with agents for a few hours. The real bottlenecks are often not the ones in the request.

2. Pick the slice

Choose where volume is high, answers exist in documents or systems, and mistakes are recoverable. For example: "order status and return-policy questions in English, which are 35% of tickets." Avoid the hardest or riskiest categories (billing disputes, legal complaints) in the first pilot.

Also choose the mode, starting with the least risky that still pays:

1. Draft replies for agents to edit (a person sends).
2. Answer automatically, with hand-off when unsure.
3. Take actions (refunds, changes) with limits and approvals.

3. Agree success criteria up front

• The primary metric with today's baseline: deflection rate, handle time, first-response time.
• Quality bar: accuracy on a test set of, say, 200 past tickets with known good answers; no policy violations.
• Guardrails: escalation rate, satisfaction must not drop.
• Written down and signed off by the sponsor, so the end-of-pilot decision is not a matter of opinion.

4. Define boundaries

Data the system may access, actions it may take, languages, hours, the hand-off rule, and who reviews failures. Bring their security and legal teams in early. They are the most common cause of delay.

5. Plan the pilot

• Week 1 to 2: data access, test set, first prototype measured on the test set.
• Week 3 to 4: draft mode with a few agents; daily review of failures.
• Week 5 onwards: wider use or limited auto-answers; weekly results against the baseline.
• Final review: results, costs, risks and the plan to scale.

Why not "an agent"?

Often a well-grounded answer bot plus a few lookups beats an autonomous agent: simpler, cheaper, easier to trust. Start simple, and add agency where the workflow needs it.

What interviewers listen for: you narrowed the scope, insisted on a baseline, and planned for the decision before writing code.