Module 19 / Algorithms behind big systems

Module 19 · lesson 19 of 22 · Read 9 min

Unique IDs at scale

UUIDs, Snowflake IDs and ticket servers.

When Tiffin ran on one Postgres database, every order got its ID from BIGSERIAL: 1, 2, 3, and so on. Simple, short, sorted by time. Then the orders table was sharded across 16 databases (Module 9), and each shard started handing out its own 1, 2, 3. Order #50211 now exists 16 times.

Every large system hits this. You need IDs that are unique across many machines, generated fast, without a phone call to some central counter for every new row. And you'd like them small and roughly sorted by time, because, as you saw in the B-trees lesson, the index likes that.

What you want from an ID

Property Why it matters
Unique, always Two orders with one ID means one overwrites or misroutes the other
No coordination per ID A central counter is a bottleneck and a single point of failure
Roughly time-ordered New rows go to the right edge of the B-tree, and "newest first" is a cheap sort
Compact 64 bits fits a bigint. 128 bits doubles every index and foreign key
Not guessable (sometimes) Sequential order IDs in URLs let anyone count your orders or try their neighbour's

No single scheme wins all five. Here are the main options, then the one most big systems end up with.