The full answer
A sensor in the default poke mode holds a worker slot for its whole wait, so 200 waiting sensors are 200 slots doing nothing. There are 2 better modes. Reschedule mode takes a slot only while it checks, then frees it until the next check. Deferrable mode hands the wait to the triggerer, a separate component where many small asynchronous triggers run in one process, so the task holds no worker slot and, by default, no pool slot while it waits. For hundreds of long waits, choose deferrable where the sensor supports it, reschedule where it does not, and set a timeout on every sensor.
1. The 3 modes
| Mode | Worker slot while waiting | Good for |
|---|---|---|
| poke (default) | Held the whole time | Short waits checked every few seconds |
| reschedule | Only during each check | Waits of minutes or hours, checked every minute or more |
| deferrable | None; the triggerer waits | Many long waits, the most efficient |
Airflow's docs put the poke and reschedule choice as latency: checking every second suits poke, every minute suits reschedule.
2. The change
from airflow.providers.amazon.aws.sensors.s3 import S3KeySensor
wait_for_file = S3KeySensor(
task_id="wait_for_partner_file",
bucket_key="s3://inbound/partner_a/{{ ds }}/orders.csv",
deferrable=True, # the triggerer waits, not a worker
poke_interval=300, # check every 5 minutes
timeout=8 * 60 * 60, # fail after 8 hours instead of waiting forever
)
For a sensor with no deferrable mode:
wait_for_file = SomeSensor(task_id="wait", mode="reschedule", poke_interval=300, timeout=8 * 60 * 60)
The operators.default_deferrable setting turns deferrable mode on for every operator that supports it.
3. What to check after
- A triggerer is running. Airflow needs at least one triggerer process, beside the scheduler, for deferred tasks to work.
- Pools. Deferred tasks do not take pool slots by default; if a pool exists to limit calls to a partner's server, set the pool to count deferred tasks.
- Timeouts. A sensor with no timeout can wait for a file that never comes; a timeout turns that into a failure someone sees.
4. A better design where possible
If the files land through a process you control, have it update an Airflow asset when it finishes and schedule the Dag on that asset (see the question on making one Dag wait for another), and there is nothing to sense.
What interviewers want: poke holding a worker slot, reschedule freeing it between checks, deferrable moving the wait to the triggerer with no worker or pool slot, the latency trade-off, timeouts on every sensor, a running triggerer, and replacing the sensor with an event where possible.
Last updated 16 October 2026
Sources