Skip to the page
Tecyfy

Databricks interview question · medium

Your Databricks bill went up 60% in a month and nobody can say why. How do you find the cause and stop it happening again?

The short answerQuery the billing system table, group the spend by SKU, workspace, compute and tags, and compare the two months until the increase sits in a few rows.

The full answer

Query the billing system table, group the spend by SKU, workspace, compute and tags, and compare the two months until the increase sits in a few rows. Most Databricks cost surprises come from a short list: scheduled jobs running on all-purpose clusters, interactive clusters or warehouses that never shut down, a job that got slower or runs more often, and oversized compute. Then put guardrails in place so the next one shows up in days, not on the invoice.

1. Break down the spend

system.billing.usage has one row per usage record, with the SKU, date, quantity, tags and the IDs of the job, cluster or warehouse:

SELECT sku_name,
       usage_metadata.job_id,
       usage_metadata.cluster_id,
       usage_metadata.warehouse_id,
       date_trunc('month', usage_date) AS month,
       SUM(usage_quantity) AS dbus
FROM system.billing.usage
WHERE usage_date >= '2026-08-01'
GROUP BY ALL
ORDER BY dbus DESC;

Join to system.billing.list_prices to turn DBUs into money, and compare month against month.

2. What to look for

FindingLikely causeFix
All-purpose compute SKU grewScheduled jobs or pipelines pointed at an interactive clusterMove them to job compute, which costs significantly less for the same work
Cluster usage around the clockNo auto-termination, or a notebook keeping it aliveAuto-termination on every interactive cluster
SQL warehouse on all dayClassic warehouse left running for dashboardsServerless warehouse with a short auto-stop
One job's DBUs doubledIt slowed down (data growth, skew, small files) or runs more oftenFix the job; check its schedule and triggers
Many large clusters from one teamNo limits on what people can createCompute policies with size limits

A useful sign: job_id is empty for jobs that ran on all-purpose compute. A lot of all-purpose usage with no job ID but a job-like daily pattern is usually scheduled work on the wrong compute.

3. Guardrails

  • Tags on clusters, warehouses and jobs (team, project, environment). Tags only apply from when they are added, so start now.
  • Compute policies that cap worker counts, restrict instance types and require auto-termination.
  • Budgets and alerts per team, with a weekly report from the billing table.
  • Review the top 10 jobs by cost each month. Those few usually make up most of the bill, and Photon, serverless or a better layout often makes them both faster and cheaper.

What interviewers want: using the billing system table rather than guessing, the common causes (all-purpose vs job compute, idle compute, slowing jobs), and tags, policies and budgets as the lasting fix.

Last updated 16 October 2026