The full answer
Add the new model as a second deployment on the same endpoint and move traffic to it in steps, checking metrics at each one: no live traffic, then mirrored traffic, then a small share of live traffic, then all of it. The endpoint's address and keys stay the same throughout, so clients change nothing, and rolling back is one traffic update.
The setup
An online endpoint is the stable address clients call. Behind it sit one or more deployments, each with its own model, environment, VM type and instance count. The endpoint's traffic setting decides which deployment answers, as percentages such as {"blue": 90, "green": 10}.
The steps
| Step | Traffic | What you check |
|---|---|---|
| 1. Deploy green | blue 100, green 0 | The deployment starts and passes its probes |
| 2. Test in isolation | Call green by name | Correct outputs on known inputs, latency |
| 3. Mirror | blue 100, green mirrored 10–50% | Latency, errors and prediction distribution on real requests |
| 4. Canary | blue 90, green 10 | Live error rate, latency, business metrics |
| 5. Switch | green 100 | Same metrics over a full business cycle |
| 6. Clean up | Delete blue | Only after you are sure you will not roll back |
endpoint.traffic = {"blue": 90, "green": 10}
ml_client.online_endpoints.begin_create_or_update(endpoint).result()
What mirroring gives you
Mirrored (shadow) traffic copies a share of real requests to green, but clients still get blue's answers. You see how green behaves on production inputs, its latency, errors and how its predictions differ from blue's, with no effect on users. The limits matter in practice:
- Only one deployment can receive mirrored traffic, at most 50%.
- A deployment gets live or mirrored traffic, not both.
- Requests that name a deployment directly are not mirrored.
- Kubernetes online endpoints do not support it.
Before you start
- Quota. Green runs alongside blue, so you need cores for both. On many VM types Azure ML also reserves 20% extra for upgrades: 10 instances of a 4-core VM need quota for 48 cores.
- Decide the pass criteria in advance: the latency and error limits, and the model metric green must match or beat. Without them, every step becomes a judgement call.
- Log predictions with the deployment name, so you can compare blue and green on the same requests.
Rolling back
If green misbehaves at any step, set traffic back to blue. Blue is still running, so this takes effect at once. That is why you delete blue last.
What interviewers want: an endpoint with several deployments, traffic moved in steps, mirroring for risk-free testing on real traffic and its limits, quota for running both, and rollback through traffic rather than redeploying.
Last updated 16 October 2026
Sources