The full answer
ORDER BY produces a total order of the whole result. SORT BY only sorts rows within each partition, so the overall output may be out of order. CLUSTER BY col is shorthand for DISTRIBUTE BY col SORT BY col: it repartitions rows so equal values go to the same partition, then sorts within each partition.
-- 1. Total order: one global sort
SELECT * FROM sales ORDER BY amount DESC;
-- 2. Sorted within each partition only
SELECT * FROM sales SORT BY amount DESC;
-- 3. Same keys together, sorted within partitions
SELECT * FROM sales CLUSTER BY customer_id;
-- same as
SELECT * FROM sales DISTRIBUTE BY customer_id SORT BY customer_id;
ORDER BY | SORT BY | DISTRIBUTE BY | CLUSTER BY | |
|---|---|---|---|---|
| Shuffle | Yes: range partitioning across the cluster | No | Yes: hash partitioning by the expressions | Yes: hash partitioning |
| Sort | Global | Within each partition | None | Within each partition |
| Output fully ordered | Yes | Only with one partition | No | No |
| Directions | ASC or DESC per column | ASC or DESC per column | Ascending only | |
| DataFrame API | orderBy, sort | sortWithinPartitions | repartition(col) | repartition(col).sortWithinPartitions(col) |
How ORDER BY works
Spark samples the data to choose range boundaries, shuffles rows into ordered ranges, and sorts each range, so partition 1 holds the smallest values. It is the most expensive of the three; with LIMIT, Spark uses a cheaper top-N sort.
When to use each
ORDER BYfor final results that people read or that must be ordered, preferably after aggregation, when data is small.SORT BYwhen only per-partition order matters, such as preparing sorted output files where each file is sorted.DISTRIBUTE BYorCLUSTER BYto group all rows with the same key in one partition before per-key processing, or so each written file contains whole keys, which improves compression and data skipping.
Pitfalls
- Don't rely on
SORT BYorCLUSTER BYfor an ordered report: the output is only ordered within partitions. - Ordering isn't preserved by later shuffles; sort as the last step.
CLUSTER BYin aSELECTis unrelated toCLUSTER BYinCREATE TABLE, which (for Delta and some other table formats) defines liquid clustering, a storage layout.
Last updated 16 October 2026
Sources