How does Kafka keep messages in order, and what breaks that order? A team wants every event for one customer processed in the order it happened, and plans to add partitions later as traffic grows.

Kafka orders messages within a partition, never across a topic. Give every event the customer's id as its key: the producer sends all records with the same key to the same partition (a hash of the key), and every consumer of that partition reads them in the order they were written. Two things break it: adding partitions, which moves keys to other partitions, and retries without idempotence, which can swap 2 batches. So size the partitions for the traffic expected in 1 to 2 years at the start, and keep the producer idempotent (the default).

1. Where the order lives

producer.send("customer-events", key=customer_id.encode(), value=event)

A record with no key is spread over partitions for throughput, so 2 events from one customer can land in different partitions and be read in either order.

2. Why adding partitions breaks it

The partition is the hash of the key modulo the number of partitions. Change the number and many keys map somewhere new. Kafka does not move existing data, so for a while a customer's old events sit in partition 3 and new ones in partition 7, read by different consumers at different speeds: an update can be processed before the event it follows.

Avoid it by choosing the count up front, from the throughput a partition takes from producers and gives a consumer, sized for the future traffic.

3. Why retries can reorder

If a batch fails and is retried while the next batch to the same partition succeeds, the second batch is written first. The idempotent producer (on by default, with acks=all, retries above 0 and at most 5 requests in flight) keeps the order. Turn idempotence off and keep several requests in flight, and order is no longer guaranteed.

4. Inside the consumer

Order holds as far as the consumer's own code keeps it. A consumer that hands records from one partition to a thread pool processes them out of order. Keep one key's records on one thread, for example by hashing the key to a worker.

What interviewers want: ordering per partition and per key, the key choosing the partition, why adding partitions moves keys, the idempotent producer keeping order through retries, and parallel processing inside the consumer as the last place order is lost.