To save storage, someone ran `VACUUM ... RETAIN 0 HOURS` on a production Delta table. Now a long-running job failed with missing files, and you cannot time travel to yesterday. What happened, and what do you do?

VACUUM deleted every data file that the current version of the table no longer references. That included the files older versions point to, so time travel to them is gone, and the files that running jobs were still reading or had written but not yet committed. Recover what you can from the current version and other copies, rerun the failed job, and make sure a retention below 7 days cannot be set again.

Why it broke

A Delta table's history is a log of versions, each pointing to a set of data files. An UPDATE or MERGE does not change files in place; it writes new ones and marks the old ones as removed. Those old files are what time travel and RESTORE read. VACUUM deletes removed files older than the retention period, 7 days by default. With 0 hours:

• Time travel stops working for every earlier version. The log still lists them for 30 days by default, but the data files are gone, so queries and restores fail.
• Running jobs break. A query that started on an older snapshot may need files that were just deleted. A long write may have written files it had not yet committed; VACUUM saw them as unreferenced and removed them.

The safety check retentionDurationCheck exists to block this; someone turned it off.

What to do now

1. Stop scheduled VACUUM on the table and check the table's history for the VACUUM operation, its user and time.
2. Rerun the failed job. It starts from the current version, which is intact.
3. Recover lost versions only from other copies: cloud storage versioning or soft delete if it was enabled on the bucket or container, a deep clone, or a backup. If none exists, earlier versions cannot be restored.
4. Tell the people affected that rollbacks and audits on that table cannot reach before the VACUUM.

Prevent it

• Keep retention at 7 days or more, set on the table (deletedFileRetentionDuration on Databricks Runtime 18.0 and above), and longer than your longest job and your rollback needs.
• Never disable retentionDurationCheck in shared code. Restrict who can run maintenance on production tables.
• Use VACUUM ... DRY RUN to see what would be deleted before a manual run.
• On Unity Catalog managed tables, let predictive optimization run VACUUM instead of ad hoc commands.
• If saving storage was the goal, measure it first: often the bigger saving is from fewer table versions or from tables nobody reads.

What interviewers want: that time travel depends on old data files, what VACUUM deletes, why 0 hours breaks running jobs, the retention check, honest recovery options, and prevention through table settings and permissions.