The full answer
Databricks scales compute in three ways: classic clusters add and remove worker nodes with autoscaling, SQL warehouses add and remove whole clusters as queries queue up, and serverless compute is scaled by Databricks with no settings at all.
1. Classic clusters: autoscaling workers
You set a minimum and maximum number of workers. Databricks adds workers when tasks are waiting and removes them when they are idle.
- Optimized autoscaling (Premium plan and above) goes from minimum to maximum in at most two steps, and can scale down even while the cluster is busy by checking which workers hold shuffle files still needed. On jobs compute it scales down after 40 seconds of low use; on all-purpose compute after 150 seconds.
- Standard autoscaling (the standard plan) starts by adding 8 nodes and grows exponentially, and scales down only when 90% of nodes have not been busy for 10 minutes.
- Autoscaling local storage attaches more disk to a worker before it runs out of space.
- Auto termination stops an idle all-purpose cluster entirely after the minutes you choose; jobs compute ends with its run.
Autoscaling suits workloads that vary. Streaming jobs on classic compute are usually better with a fixed size, or with Lakeflow pipelines, which have their own autoscaling for streaming.
2. SQL warehouses: more clusters for more queries
A warehouse has a size (2X-Small up to 5X-Large), which sets how big each cluster is, and a range of clusters.
- Serverless warehouses use Intelligent Workload Management: each query's needs are predicted, queries are queued if capacity is short, and clusters are added quickly as wait times rise.
- Pro and classic warehouses add clusters based on the estimated load of running and queued queries, scale up if a query waits 5 minutes, and scale down after 15 minutes of low load.
The size helps single large queries; more clusters help many concurrent users.
3. Serverless compute for notebooks, jobs and pipelines
Databricks runs the machines, starts them in seconds and scales them to the work. There is no cluster to size, and you pay only while work runs.
4. Choosing machines
On classic compute you still choose worker types: memory-optimised for large joins and caching, compute-optimised for CPU-heavy transformations, GPU instances for deep learning, and Photon for faster SQL and DataFrame work.
The result is that capacity follows demand: more machines during heavy work, few or none when idle, and less time spent sizing clusters by hand.
Last updated 16 October 2026
Sources