The full answer
Match each to how requests arrive and how big and slow they are: a real-time endpoint for the fraud score, Serverless Inference for the occasional classifier, Asynchronous Inference for the long document job, and batch transform for the nightly churn scores. The deciding facts are latency needs, payload size, processing time and how much of the day the model sits idle.
The 4 options side by side
| Option | Best for | Limits that decide it | Cost when idle |
|---|---|---|---|
| Real-time endpoint | Interactive calls that need an answer in milliseconds | Request body up to 6 MB, response within 60 seconds | You pay for the instances all the time |
| Serverless Inference | Spiky or rare traffic that can accept a cold start | 1 to 6 GB of memory, no GPU, no VPC or data capture | Scales to 0 |
| Asynchronous Inference | Large inputs or long processing, answer wanted soon but not at once | Payload up to 1 GB, up to 1 hour per request, input and output through S3 | Can scale to 0 |
| Batch transform | A whole dataset at once, on a schedule | No endpoint; a job that starts, scores files in S3 and stops | Nothing between jobs |
1. Fraud score at checkout: real-time endpoint
A checkout cannot wait for a cold start or a queue. Put it on instances that are always running, with autoscaling to follow the day's traffic. It also keeps options the other choices drop: data capture for monitoring, several production variants for testing a new model against the old one, and a VPC.
2. Classifier used a few dozen times a day: Serverless Inference
A real-time instance would sit idle almost all day and still be billed. Serverless scales to 0 and charges per use. Pick the memory size from the model's size and benchmark latency at each size, since vCPUs grow with memory. Watch OverheadLatency in CloudWatch: if the first request after a quiet spell is too slow for users, add Provisioned Concurrency for the hours people work. Check the exclusions first: no GPU and no VPC configuration, so a model that needs either goes on a real-time endpoint instead.
3. 300 MB files taking 10 minutes: Asynchronous Inference
Both limits rule out real-time: the input is far over 6 MB and the work is far over 60 seconds. Asynchronous Inference takes the input as an S3 location, queues the request, writes the result to S3 and can send an SNS notification on success or failure. Let autoscaling take it to 0 instances when no requests are waiting, so the quiet hours cost nothing.
4. Nightly churn scores for every customer: batch transform
Nobody calls this model; a schedule does. Batch transform reads the customer file from S3, spreads it across instances, writes predictions back to S3 and shuts down, so nothing is billed during the day. Splitting the input into several files lets more than one instance work at once.
Mistakes to avoid
- Calling a real-time endpoint in a loop for a nightly job: you pay for instances sized for the peak and hit throttling.
- Choosing serverless for a model that needs a GPU or must run inside a VPC.
- Picking real-time for a long job and raising client timeouts: the container still has to answer within 60 seconds.
What interviewers want: the deciding questions (latency, payload size, processing time, idle time), the real limits behind each option, and the trade-offs you accept, such as cold starts on serverless or the lost monitoring features.
Last updated 16 October 2026