How do you decide whether a customer's use case is a good fit for an LLM at all?

A use case fits an LLM when the work involves language or unstructured information, a good-enough answer is valuable even if not perfect, mistakes can be caught or are cheap, and there is enough volume to repay the effort. It is a poor fit when a rule, a query or ordinary software can do the job exactly, when every answer must be 100% right with no review, or when the data the model would need does not exist or cannot be shared.

Questions I ask

1. What is the input and output? Emails, documents, conversations, code, free text: good fit. Numbers in a table that need an exact calculation: use code (or let the model call code).
2. Is there a right answer, and can we check it? If correct output can be verified (by code, a person, or a downstream system), we can measure and improve.
3. What does an error cost? A wrong draft that an agent edits costs seconds. A wrong medication dose or a wrong payment is serious, so that needs a person in the loop or a different solution.
4. How is it done today, and how much of it? Volume times time per item times cost per hour gives the size of the prize. Low-volume work rarely repays a careful build.
5. Is the needed knowledge available? In documents, systems or people's heads? If only in heads, the first project is capturing it.
6. Can the data be used? Privacy, contracts, regulation, data residency.
7. Would a simpler tool do? A search box, a form, a rule engine or a report may solve it more reliably.

Good fits (typical)

• Drafting replies, summaries and reports for a person to review.
• Extracting fields from messy documents into a system.
• Classifying and routing requests.
• Search and question answering over internal documents.
• Coding help for internal teams.

Poor fits (typical)

• Exact arithmetic or accounting without tools.
• Decisions with legal effect on people without human review.
• Tasks needing knowledge that exists nowhere in writing.
• Very low volume or one-off work.

How to say no well

Do not just refuse. Offer the better option ("this is a rules problem; a validation step in your existing system will be exact and cheaper"), or a smaller AI piece that does fit ("the model can draft the explanation; the calculation stays in your system").

Check quickly

A one-day prototype on 30 real examples often settles the question faster than a week of debate. If it does well on easy cases and fails on the important ones, that is your answer.

What interviewers want: judgement. Telling a customer where AI will not help builds more trust than promising it will help everywhere.