Skip to the page
Tecyfy

Azure interview question · hard

An Azure Data Factory pipeline runs 5 copy activities in sequence, with a logging activity on the Upon Failure path of the last one. On Monday copy 2 failed and nothing was logged. On Tuesday copy 5 failed, the error was logged, but the run showed Succeeded and the alert on failed runs stayed silent. Explain both nights and fix the design.

The short answerOn Monday, copies 3 to 5 were skipped after copy 2 failed, so copy 5 never failed and its Upon Failure path never ran.

The full answer

On Monday, copies 3 to 5 were skipped after copy 2 failed, so copy 5 never failed and its Upon Failure path never ran. On Tuesday the design behaved as a Try-Catch: copy 5 failed, the handler succeeded, and ADF marks such a run Succeeded, so the failed-run alert had nothing to fire on. Fix it with the generic error handling pattern (connect both Upon Failure and Upon Skip from the last activity to the handler), end the handler with a Fail activity so the run is marked failed and alerts fire, and add retries for transient errors.

1. The 4 paths

PathRuns when the activity before it
Upon Successsucceeded
Upon Failurefailed
Upon Completionsucceeded or failed
Upon Skipdid not run at all

A handler on Upon Failure of copy 5 is only for copy 5's own failure. When anything earlier fails, copy 5 is skipped, not failed.

2. How ADF decides the run's status

It looks at the leaf activities. A skipped leaf is replaced by its parent, up the chain. The run is Succeeded only if every node evaluated succeeded. That is why the 3 well-known shapes give different results:

PatternPaths definedIf the activity fails, the run shows
Try-CatchUpon Failure onlySucceeded (the handler succeeded)
Do-If-ElseUpon Success and Upon FailureFailed
Do-If-Skip-ElseBoth, plus a dummy on Upon SkipSucceeded

Tuesday's pipeline was a Try-Catch. Whether you want Succeeded after a handled failure is a choice; for a load whose data is now incomplete, you almost always want Failed.

3. The fixed pipeline

Copy1 → Copy2 → Copy3 → Copy4 → Copy5
                                   ├─ Upon Failure ─┐
                                   └─ Upon Skip ────┴→ LogError → FailRun
  • LogError (a stored procedure or web call) records @pipeline().RunId, @pipeline().Pipeline and the error of the failed activity.
  • FailRun is a Fail activity with a message and error code built from that error. The run now shows Failed, so the Azure Monitor alert on failed pipeline runs fires, and an orchestrating pipeline that calls this one sees the failure.

Because Upon Skip also fires when copy 5 is skipped for any earlier failure, one handler covers all 5 copies.

4. Handle transient errors before they become failures

Set a policy on each copy:

"policy": { "retry": 3, "retryIntervalInSeconds": 120, "timeout": "02:00:00" }

Retry defaults to 0, so a brief network error fails the run without these settings. The default timeout is 12 hours: a hung copy can hold the pipeline for half a day before anyone hears, so set a timeout near what the copy normally takes, with headroom.

5. Make reruns safe

After a failure someone reruns the pipeline (or from the failed activity). Each copy should overwrite its target partition or folder, or load into a staging table and swap, so running copy 1 again does not duplicate yesterday's rows.

What interviewers want: the difference between failed and skipped, how ADF derives the run status from leaf activities, Upon Failure plus Upon Skip for one shared handler, a Fail activity so monitoring sees the failure, retry and timeout policies, and idempotent reruns.

Last updated 16 October 2026