A nightly SageMaker batch transform job scores one 40 GB CSV of customers on 10 instances and takes 6 hours. CloudWatch shows 1 instance busy and 9 idle, and the output has no customer id, so nobody can join the scores back. How do you fix both problems?

Batch transform hands out work by file, so 1 input file means 1 busy instance. Split the input into many files (at least as many as instances, better several per instance), split each file into mini-batches by line, and set the concurrency to the workers per instance. For the missing ids, send the model every column except the id with InputFilter, join the input back with JoinSource, and keep only the id and the score with OutputFilter.

1. Why 9 instances sat idle

Batch transform maps S3 objects to instances by key. With one 40 GB object, one instance gets all of it; the other 9 are billed and do nothing. Write the export as many files instead, for example 200 files of about 200 MB under one prefix. Spark, Athena UNLOAD or Glue can all write a partitioned output.

2. The job settings

TransformInput={
    "DataSource": {"S3DataSource": {"S3DataType": "S3Prefix", "S3Uri": "s3://ml/churn/input/2026-10-16/"}},
    "ContentType": "text/csv",
    "SplitType": "Line",            # mini-batches of lines, not the whole file as 1 request
},
BatchStrategy="MultiRecord",        # several lines per request, up to MaxPayloadInMB
MaxPayloadInMB=6,
MaxConcurrentTransforms=4,          # the workers in each instance; 4 x 6 MB stays under 100 MB
TransformOutput={"S3OutputPath": "s3://ml/churn/scores/2026-10-16/", "AssembleWith": "Line", "Accept": "text/csv"},
TransformResources={"InstanceType": "ml.m5.xlarge", "InstanceCount": 10},

The limits: MaxPayloadInMB at most 100, and MaxConcurrentTransforms × MaxPayloadInMB at most 100. The best MaxConcurrentTransforms is the number of workers in the container. Without SplitType set to Line, each file is sent as one request, which fails or crawls on large files. Without AssembleWith, output records are concatenated in binary rather than one per line. CSV fields with line breaks inside them break the splitting, so clean or quote them out in the export.

3. Getting the customer id back

Say each row is customer_id, feature_1, ..., feature_40:

DataProcessing={
    "InputFilter": "$[1:]",       # send everything after the id to the model
    "JoinSource": "Input",        # join each prediction to its full input row
    "OutputFilter": "$[0,-1]",    # keep the id (first) and the score (last)
},

Each output line is then customer_id, score. CSV rows are treated as JSON arrays, so only index paths work. Predictions stay in input order within each file anyway, but joining by position across files is fragile; the id in the output is what makes the scores safe to load.

4. Check the gain

After the change, all 10 instances should show load in CloudWatch. If the job is still slow with all instances busy, the cost is the model itself: try a larger instance type, or fewer, larger instances, and compare the total instance-hours rather than the wall-clock time alone.

What interviewers want: knowing that work is distributed per S3 object, SplitType, BatchStrategy, MaxPayloadInMB and MaxConcurrentTransforms and their limits, AssembleWith, and the InputFilter/JoinSource/OutputFilter trio for keeping ids with predictions.