How do you choose which model to use for a new LLM feature?

Decide what the feature needs (quality bar, latency, cost per request, context length, input types, data and hosting rules), shortlist two or three models that meet the hard constraints, and run your evaluation set on each. Pick the cheapest and fastest one that clears the quality bar, and keep the choice easy to change, because models are replaced every few months.

Hard constraints first

• Data and hosting: may the data go to this provider? Is it available in the cloud and region you need (the provider's API, Bedrock, Vertex AI, Azure)?
• Inputs: images, PDFs, audio, very long documents.
• Features: tool use, structured outputs, citations, batch processing, caching.
• Context window: enough for your largest prompt plus output.

Then the trade-off: capability, speed, cost

Anthropic's model-choice guide frames it the same way and offers two starting points:

• Efficiency-first: start with a fast, cheap model and move up only where your evaluations show it falls short. Good for high-volume, simpler tasks (classification, extraction, routing).
• Capability-first: start with the most capable model to learn what is possible, then try cheaper models and keep them where quality holds.

The same guide notes that adjusting the effort setting within one model is often a better lever than switching models.

Measure on your own task

• Public benchmarks and leaderboards help you shortlist; they do not tell you how a model does on your documents and users.
• Compare quality per category, cost per successful task (retries and escalations included), and p95 latency.
• Include safety and refusal behaviour, and output length, which changes cost.

Tecyfy's model finder and cost calculator compare current prices and capability scores to help with the shortlist.

Use more than one model

Many products route: a small model for classification and easy questions, a larger one for hard cases. The router's mistakes are part of the evaluation.

Keep it changeable

• Call models through one small interface, with the model id in configuration.
• Pin versions, and record the model with every output.
• Providers retire models on a schedule, so re-running the evaluation on new models is routine work.

What interviewers want: constraints, then evidence from your own evaluation, then the cheapest option that works, not "the best model on the leaderboard".