Module 14 / Partitioning

Module 14 · lesson 3 of 7 · Read 7 min

Partitioning by hash

Even spread, lost ranges.

Range partitioning keeps neighbours together. That's its strength, and it's also why a run of similar keys, like "everything from the last second", can pile onto one machine.

Hash partitioning does the opposite on purpose. It scatters keys so that neighbours end up far apart. Load spreads evenly almost for free. The price is that "give me everything between X and Y" stops being cheap.

How it works

Run the partition key through a hash function. A good hash function turns similar inputs into wildly different outputs that look random and are spread evenly across the output range.

hash("order-1000001") = 0x3f9a12c4
hash("order-1000002") = 0xd1076b0e
hash("order-1000003") = 0x5c22f8a1

Orders 1000001 and 1000002 are neighbours, but their hashes aren't. Now assign partitions by hash range. With a 32-bit hash and four partitions:

Partition Hash range
P1 0x00000000 to 0x3fffffff
P2 0x40000000 to 0x7fffffff
P3 0x80000000 to 0xbfffffff
P4 0xc0000000 to 0xffffffff

order-1000001 goes to P1, order-1000002 to P4, order-1000003 to P2. Orders created one after another land on different partitions. The "everything new goes to the last partition" problem from range partitioning simply goes away.

Hash partitioning assigns each key to a partition based on a hash of the key, so keys spread evenly but lose their natural order.

New ordersIDs going up
→
Hash functionscrambles the key
→
P1~25%
P2~25%
P3~25%
P4~25%
Hashing turns keys that arrive in order into keys spread evenly across partitions.