Bitly: URL shortener

A URL shortener is the "hello world" of system design, and that's exactly why it's worth doing properly. The product fits in one sentence, so all your attention can go on the parts that are actually hard: making a short code that never collides, and answering billions of redirects fast enough that nobody notices you exist.

You'll build this from a blank page. At each step, try to decide before you read on.

The brief

You're building a service like Bitly. Someone pastes a long URL, gets back something like sho.rt/aZ3kQ9x, and shares it. Anyone who opens the short link lands on the long URL.

Who uses it?

• Creators (marketers, newsletter writers, apps that send SMS) make links, often through an API.
• Clickers are everyone else. They never sign in, never see your UI, and only notice you if you're slow or broken.

That second group shapes almost everything. The clicker's experience is one HTTP redirect, and it has to be fast.

In scope for this lab:

• Create a short link, optionally with a custom alias.
• Redirect a short link to its long URL.
• Count clicks per link.
• Links can expire.

Out of scope: user accounts and billing, QR codes, link editing UI, spam and phishing detection (important in real life, but a separate lab), and detailed analytics dashboards.

Requirements

Functional

• POST a long URL, get back a generated short code of at most 7 characters.
• Optionally ask for a custom alias like sho.rt/diwali-sale instead. Aliases can be longer than 7 characters. If it's taken, say so.
• GET sho.rt/<code> redirects to the long URL.
• A link can have an expiry time. After it, the short link returns 404.
• Each link has a click count, accurate to within a minute or so.

Non-functional

• Redirects are fast: p99 under 50 ms at the server, not counting the user's network.
• Redirects are highly available: 99.99%. A dead short link in a printed poster can't be fixed.
• Codes never collide. Two different long URLs must never get the same code.
• Links are durable. Once created, a link works for years.
• Creating links can be a little slower (a few hundred ms is fine) and can tolerate brief outages better than redirects can.

Notice the asymmetry. Reads and writes have very different requirements, and you'll design them differently.

Numbers

Assume a mid-sized service:

• 100 million new links a month.
• 100 clicks for every link created (most links get a handful, a few get millions).

Writes:

• 100M / (30 × 86,400 s) ≈ 100M / 2.6M ≈ 40 writes/s on average.
• Peak at 5× average: ~200 writes/s.

Reads:

• 40 × 100 = 4,000 redirects/s on average.
• Clicks are spikier than creates, because one viral link can bring a million people in an hour. Use 10× for peak: ~40,000 redirects/s.

Storage:

• Per link: code (7 B), long URL (average ~200 B, max 2 KB), owner, created and expiry times, a few flags. Call it ~500 bytes with indexes.
• Per year: 100M × 12 = 1.2 billion links × 500 B = ~600 GB a year.
• Keep links for 10 years: ~6 TB. That's more than you'd like on one database, but not by a crazy amount.

Code space:

• Base62 (a-z, A-Z, 0-9) with 7 characters: 62⁷ ≈ 3.5 trillion codes.
• 1.2 billion a year uses that up in about 3,000 years. Seven characters is plenty.

Bandwidth:

• A redirect response is tiny, ~500 bytes with headers. 40,000/s × 500 B = 20 MB/s outbound at peak. Not a concern.

Cache sizing:

• Clicks follow a steep power law. Say 20% of links get 80% of clicks, and only links from the last few months are hot. Caching the 100 million most recent links at ~500 B each is ~50 GB, which fits in a small Redis cluster.

The numbers tell you the story: writes are trivial, reads are 100× bigger and spiky, and the data is medium-sized. This is a caching problem wearing a database costume.

API

POST /api/links
Content-Type: application/json
Idempotency-Key: 6f1c...

{ "long_url": "https://example.com/sale?utm=sms", "alias": "diwali-sale", "expires_at": "2027-01-01T00:00:00Z" }

201 Created
{ "code": "diwali-sale", "short_url": "https://sho.rt/diwali-sale" }

409 Conflict     (alias already taken)

GET /{code}

302 Found
Location: https://example.com/sale?utm=sms

404 Not Found    (unknown or expired)

GET /api/links/{code}/stats

200 OK
{ "clicks": 18234, "updated_at": "2026-10-18T10:04:00Z" }

Two choices to notice:

• The redirect lives at the root path /{code}, not under /api, because the whole point is a short URL.
• The idempotency key on create means an SMS app that retries after a timeout doesn't make 3 links for one message. Module 12 covers how idempotency keys work.

Data model

There's one core lookup: code to long URL. Everything else is secondary. The schema is written as SQL for readability; the same shape works as items in a key-value store.

CREATE TABLE links (
  code        VARCHAR(32) PRIMARY KEY,   -- 7 chars generated, longer for custom aliases
  long_url    TEXT        NOT NULL,
  owner_id    BIGINT,
  created_at  TIMESTAMP   NOT NULL,
  expires_at  TIMESTAMP,
  is_custom   BOOLEAN     NOT NULL DEFAULT FALSE
);

CREATE TABLE click_counts (
  code        VARCHAR(32) PRIMARY KEY,
  clicks      BIGINT      NOT NULL DEFAULT 0
);

Which store? The access pattern is pure key-value: get by primary key, insert with "fail if the key exists". There are no joins on the hot path. So you have two good options:

Either is a fine answer. What matters is that you can say why: the hot path is a single-key read, and the uniqueness check is a single-key conditional write. This lab uses a key-value store, because 6 TB that only ever grows and is only read by key is exactly what they're built for. Module 2 covers choosing between SQL and NoSQL.

Click counts live apart from links on purpose. Links are written once and read constantly. Counts are written constantly. Mixing them would turn every cached, immutable link row into a row that changes on every click.

High-level design

%% caption: Redirects, creates and click counting run as separate services, because their requirements differ.
flowchart TD
  CL([Clicker]) --> CDN["CDN edge<br/>caches popular redirects"]:::hot
  CDN --> LB[Load balancer]
  CR([Creator]) --> LB
  LB --> RS["Redirect service<br/>stateless"]
  LB --> CS["Create service<br/>validates, assigns code"]
  RS --> RC[("Redis<br/>code to URL")]:::hot
  RS -- "on a miss" --> LS[("Links store<br/>key-value, by code")]:::store
  CS --> LS
  CS -. "only with option 3" .-> IDA["ID allocator<br/>hands out ranges"]
  RS -. "click event" .-> CQ[["Click queue<br/>Kafka"]] --> CW[Count workers] --> CC[(Click counts)]:::store

Trace each box back to a requirement:

• Redirect service and create service are separate. Redirects need 99.99% and p99 under 50 ms. Creates don't. Splitting them means a bad deploy or a spike in creates can't hurt redirects, and you can scale them independently.
• CDN edge. A viral link can bring tens of thousands of clicks a second to one code. Serving that from edge caches near the user keeps it off your servers entirely and helps the latency target.
• Redis cache. Covers the long tail the CDN doesn't hold, and keeps the 40,000/s peak off the store.
• Links store. Durability and the "codes never collide" requirement, via a conditional insert.
• Click queue and count workers. Click counting is a requirement, but it must not slow the redirect. The redirect service fires an event and moves on.
• ID allocator (dotted). Codes must never collide, and generating them well is the first deep dive. The allocator is only needed for one of its options, range allocation. With random codes, the recommended default, there's no allocator at all.

Deep dive: generating codes that never collide

You need a 7-character code for each new link. There are three common approaches.

Option 1: hash the URL

Take md5(long_url), base62-encode it, keep the first 7 characters.

• Same URL gives the same code, which is nice for deduplication.
• But 7 characters of a hash will collide eventually. By the birthday paradox, with 3.5 trillion possible codes, you expect your first collision after roughly √3.5T ≈ 1.9 million links. You'll make that many in under a day. So you need "check, and if taken, add a salt and try again", which means a read before every write, and a race between two creators.
• Two different users shortening the same URL often want different links, with separate click counts. So the dedup feature isn't even what you want.

Option 2: random codes

Pick 7 random base62 characters, then insert with "fail if exists". If it fails, pick again.

• With 1 billion codes used out of 3.5 trillion, the chance a random code is taken is about 1 in 3,500. Retries are rare.
• The conditional insert makes it correct even with two creators racing for the same code.
• Codes are unguessable, so nobody can walk through aaaaaa1, aaaaaa2... and scrape every link.

This is a perfectly good answer, and simple.

Option 3: a counter, encoded in base62

Keep a global counter. Each new link takes the next number, and you base62-encode it. Number 1,000,000,000 becomes a 6-character code.

• Never collides, by construction, so no retry loop.
• But one global counter is a single point of failure and a bottleneck. And sequential codes are guessable.

The usual fix is range allocation. The ID allocator (a tiny service backed by a strongly consistent store like Postgres or ZooKeeper) hands each create server a block of 10,000 numbers at a time. The server uses them locally from memory, and asks for a new block when it runs out. The allocator is hit once per 10,000 links per server. Even if one server took all 40 writes/s, that is one call every 4 minutes or so. If a server crashes, the rest of its block is simply wasted, which is fine with 3.5 trillion to spare.

To fix guessability, run the number through a reversible scramble with a secret key before encoding: a small block cipher whose inputs and outputs are both numbers below 62⁷. A few rounds of a Feistel network do it: each round splits the number into two halves and mixes one half with a keyed hash of the other, and each round can be undone with the key. Format-preserving encryption (FF1 is the standard one) packages exactly this for 7 base62 digits. For example, numbers 1,000,000 and 1,000,001 might come out as q8Xz01B and 3MaT7cK: neighbours go in, unrelated codes come out. The result still never collides, because the scramble is a one-to-one mapping, and without the key nobody can work backwards.

Avoid the tempting shortcut of multiplying by a large constant modulo 62⁷. It's also one-to-one, but it's linear: the difference between two consecutive codes is the constant, so anyone who creates two links in a row can recover it and walk the whole sequence. Module 19 covers ID schemes like this, including Snowflake IDs.

Which one?

Random codes with a conditional insert is the simplest correct choice. Range allocation with scrambling is what you'd pick if you didn't want writes to depend on a conditional insert, or if your store can't do one cheaply.

Custom aliases go through the same conditional insert. If diwali-sale exists, the insert fails and you return 409. One rule: generated codes are exactly 7 characters from base62 (pad shorter numbers with leading zeros), and custom aliases must be either a different length or contain a -, so the two namespaces can never fight over a code.

Deep dive: making redirects fast

The redirect is the whole product for 99% of users. Here's the path of one click.

%% caption: Three layers. Most clicks stop at the first or second.
sequenceDiagram
  autonumber
  participant B as Browser
  participant E as CDN edge
  participant R as Redirect service
  participant C as Redis
  participant S as Links store
  B->>E: GET /diwali-sale
  alt cached at the edge
    E-->>B: 302 to the long URL
  else edge miss
    E->>R: GET /diwali-sale
    R->>C: look up the code
    alt Redis hit, under 1 ms
      C-->>R: long URL and expires_at
    else Redis miss
      R->>S: read by code
      S-->>R: long URL and expires_at
      R->>C: store it, TTL about a day
    end
    R-->>E: 302, s-maxage=60
    E-->>B: 302 to the long URL
  end

Layer 1: the CDN

You told browsers not to cache (302 with no max-age). But you can let your own CDN cache the 302 for a short time, say 60 seconds, with Cache-Control: s-maxage=60. Now a link getting 50,000 clicks a second sends your servers about one request per edge location per minute. The CDN takes the spike.

The cost: clicks served from the CDN never reach your redirect service, so you can't count them there. Two ways to handle it:

• Use the CDN's own request logs, shipped to you every minute, as the click source.
• Or run a small piece of code at the edge (most CDNs offer this) that serves the cached redirect and fires a click event.

Either way, "accurate to within a minute" is what lets you do this. If the requirement were "exact count, instantly", you couldn't put a CDN in front.

Layer 2: Redis

The redirect service checks Redis with the code as the key. Cache-aside (Module 3): on a miss, read from the store and put it in Redis with a TTL of a day or so.

Since a link never changes after it's created, invalidation is easy. The only change is expiry, and you handle it by storing expires_at with the cached value and checking it on every hit. An expired link is treated as a miss and returns 404, so you never need to hunt down cache entries when something expires.

Layer 3: the store

A single-key read on a key-value store is a few milliseconds. With ~95% of clicks served by the CDN or Redis, the store sees perhaps 2,000 reads a second at peak. Easy.

Lookups for codes that don't exist

Bots try random codes. Each one misses the CDN, misses Redis, and hits the store. Cache the "not found" too: store a small NONE marker in Redis with a short TTL, say 5 minutes. If attackers spray millions of unique random codes, a Bloom filter (a compact in-memory bit array that answers "definitely not here" or "maybe here", Module 19) of all existing codes in front of the store answers "definitely not" from memory. Module 3 covers this "lookups for things that don't exist" problem.

Deep dive: counting clicks without slowing redirects

The redirect service must not write to a database on every click. At 40,000 clicks a second, that's 40,000 writes a second, many of them to the same hot row for a viral link.

Instead:

1. On each redirect, the service publishes {code, timestamp, country, referrer} to a Kafka topic, without waiting for an acknowledgement on the request path. A local buffer batches events and sends them every few milliseconds.
2. Count workers consume the topic, keep counts in memory per code for 30 seconds, then write one UPDATE click_counts SET clicks = clicks + n per code.
3. A viral code getting 50,000 clicks in 30 seconds becomes one write, not 50,000.

Partition the topic by code, so every click for one code goes to the same worker and you never have two workers fighting over one row.

What about losing clicks? If a redirect server crashes with events still in its buffer, you lose a few milliseconds' worth. That's acceptable for a count that's allowed to be "about right within a minute". If the business sold click counts as billing data, you'd change that decision and wait for Kafka to acknowledge, at the cost of a little latency.

Under load: what breaks

Now put it under pressure. Imagine a celebrity tweets a short link to 100 million followers.

1. One hot key. 200,000 clicks a second on one code. The CDN absorbs nearly all of it, if the link is cacheable at the edge. In the first second, before the CDN has it, every edge location misses at once and hits your redirect service: a small stampede. Fix: request coalescing at the CDN (most support it, so one miss per edge fetches and the rest wait), and the same in Redis, where only one request per code loads from the store and the rest wait for it. Module 3 covers cache stampedes.

2. Redis goes down. Now every click that misses the CDN goes straight to the store, maybe 20× its normal read load. Fix: run Redis with replicas, and put a small in-process cache (a few hundred MB, 30-second TTL) inside each redirect server, so even a full Redis outage only multiplies store load by a little. Also set the store's read capacity so it can survive a few minutes without Redis.

3. Kafka is slow or down. The redirect service's event buffer fills up. If publishing blocks the redirect, an analytics problem becomes an outage. Fix: the buffer is bounded and drops events when full, and you alert on the drop rate. Redirects keep working. Module 18 calls this degrading gracefully.

4. The ID allocator is down. With random codes, there is no allocator, so nothing breaks. With range allocation, each server has up to 10,000 IDs in hand, so creates keep working for minutes. Give servers a bigger block, or fetch the next block when the current one is half used, so a short outage is invisible.

5. A creator floods the API. Some customer's script creates 50,000 links a second by mistake. Writes share the store with reads. Fix: per-API-key rate limits on the create service (Module 8), and separate capacity for the create path so it can't starve redirects.

6. A region goes down. Redirects need 99.99%. Run redirect services and read replicas of the links store in at least two regions, with DNS or anycast routing clickers to the nearest healthy one (Module 18). Creates can go to one primary region. A link created a second ago might not redirect in another region yet, so the create response can wait until the link is replicated, or the redirect service can fall back to the primary region on a miss.

Check against the requirements

Build it on AWS

So far every box has been generic: "a CDN", "a key-value store", "a queue". In a real design review, or an interview where the team runs on a cloud, you're expected to name the services, and the interesting arguments happen one level down: which service, configured how, and what it costs at your traffic. This section builds the same design on AWS. Prices are us-east-1 list prices as of late 2026; treat them as rough and check the AWS pricing calculator before you quote them.

The architecture

%% caption: The serving path on AWS. CloudFront answers most redirects; the click pipeline is drawn under Argument 4.
flowchart TD
  CL([Clicker]) --> R53["Route 53"] --> CF["CloudFront<br/>caches 302s for 60 s"]:::hot
  CR([Creator]) --> APIGW["API Gateway REST API<br/>usage plan per key"]
  CF -- "cache miss" --> ALB["ALB in two Regions<br/>AWS WAF attached"]
  CF -. "real-time logs" .-> CP[["Click pipeline"]]
  ALB --> RS["Redirect service<br/>ECS on Fargate"]
  APIGW --> CS["Create function<br/>Lambda"]
  RS --> DDB[("DynamoDB links<br/>global table")]:::store
  CS -- "conditional put" --> DDB

Here is what each generic box became, and the setting that matters most:

The rest of this section is the four arguments a good reviewer would start.

Argument 1: is CloudFront in front of every redirect worth it?

The CDN felt free in the generic design. It isn't. CloudFront charges per request: $0.01 per 10,000 HTTPS requests in the US (more in Europe). At an average of 4,000 redirects a second:

4,000/s x 2.6 million s/month  = 10.4 billion requests a month
10.4 billion / 10,000 x $0.01  = ~$10,400 a month, just for CDN requests

Compare the obvious alternative: no CDN for redirects. Route 53 latency routing sends clickers to an ALB in the nearest of two Regions, and a Fargate service answers from its in-process cache or DynamoDB:

Three things are worth noticing. The CDN line dominates. Security is priced per request too, so without a CDN the WAF has to inspect every click, which closes much of the gap. And the load balancer line is a classic surprise: a redirect service's clients are mostly new browsers, so new-connection pricing dominates, not bytes.

So what does the $5,000 a month buy? Three things the requirements ask for. Viral spikes are absorbed at the edge in the first second, while Fargate needs a minute or two to add tasks. Clickers far from your two Regions save a long round trip. And the edge soaks up floods before they reach your account.

The usual answer: keep CloudFront, because availability and spikes are the requirements that can't be bought back later, and then attack the price. At ten billion requests a month you're in the range where AWS offers private pricing. CloudFront's flat-rate plans don't help here: the largest tops out at 6 billion requests a month per distribution (late 2026). But the regional design is a sound and cheaper choice for a service whose clickers are concentrated near one or two Regions, as long as you keep enough spare capacity for a spike.

Argument 2: Lambda or Fargate for the redirect service?

Lambda sounds natural for a tiny stateless function. Do the maths at the origin, where CloudFront leaves about 30% of clicks, or ~1,200 a second:

Lambda:  3.1 billion invocations x $0.20 per million         = ~$620
         + 10 ms at 256 MB on Arm, about 7.8 million GB-s     = ~$100
                                                              ~ $720 a month
Fargate: 6 tasks of 0.5 vCPU and 1 GB, across two Regions   = ~$110
         + two ALBs                                          = ~$90
                                                              ~ $200 a month

Lambda's cost grows with every request; Fargate's is mostly fixed. With these numbers the lines cross at about 850 million requests a month, roughly 330 requests a second sustained. Below that, Lambda is cheaper and has nothing to run. Above it, Fargate wins, and it also has steadier latency: a cold start of a few hundred milliseconds is invisible at 50 requests a second but shows up in a 50 ms p99 when traffic jumps.

One trap matters more than the choice itself. Don't put API Gateway in front of the redirect path. At $1 to $3.50 per million requests, it would add $3,000 to $8,000 a month for features a redirect doesn't use. If you do choose Lambda, CloudFront can call a Lambda function URL directly, secured with origin access control.

The create path is the opposite case: about 100 million creates a month, spiky, and it needs per-customer rate limits. That's exactly what Lambda plus an API Gateway REST API is for. Usage plans give each API key its own throttle and quota, which handles "a creator floods the API" with configuration rather than code. It costs about $350 a month at this volume.

Argument 3: DynamoDB, and its sharp edges

The table is as simple as the generic design promised: partition key code, and the item holds long_url, owner_id, created_at and expires_at. On-demand capacity, because creates are spiky and reads depend on CloudFront's hit rate. Creates cost about $60 a month; it's a cheap table. The interesting parts are the edges:

• "Insert if absent" is a condition, not a lock. Every put carries ConditionExpression: attribute_not_exists(code). A clash raises ConditionalCheckFailedException, and the create function picks a new random code. Failed conditional writes still consume write capacity, which doesn't matter at 1 in 3,500.
• A link created a moment ago can look missing. Default reads are eventually consistent. A creator who pastes the new link into a browser within a second can get a 404 from a replica that hasn't caught up. Fix it only where it happens: if an eventually consistent read finds nothing, retry once with ConsistentRead: true before returning 404. Real misses are rare, so the extra cost is tiny.
• TTL is not an expiry check. Set expires_at as the table's TTL attribute and DynamoDB deletes expired links for free, but only "typically within a few days". Until then, expired items still come back from reads. So the redirect service keeps checking expires_at itself, exactly as in the generic design. CloudFront may also keep serving a cached redirect for up to 60 seconds after expiry. That's the price of s-maxage=60; say so in the requirements.
• One item has a ceiling. A single partition serves at most 3,000 read units a second, which is 6,000 eventually consistent reads of a small item. A viral link is a single item. CloudFront's request collapsing, plus the redirect service's in-process cache, keeps hot links far below that. This is also why the design doesn't need Redis: with CloudFront absorbing hot links, the remaining reads are spread across many codes, and DynamoDB serves them in a few milliseconds. Add DAX, DynamoDB's own cache, only if measurements say p99 needs it. DAX even caches "not found" answers.
• Bots cost money. Every random code a bot tries misses CloudFront and costs a DynamoDB read. Ten thousand junk lookups a second would cost more than all the real traffic. An AWS WAF rate-based rule stops the bulk of it. Where you attach the WAF matters, because it charges $0.60 per million requests it inspects. On CloudFront it would inspect all 10 billion redirects, about $6,200 a month. On the origin ALBs it only sees cache misses, which is exactly where random-code sprays land, for about $1,900. One catch: behind CloudFront, the ALB sees CloudFront's address, not the clicker's, so the rule has to count by the client IP that CloudFront forwards in a header. Make sure a client can't forge that header to dodge the limit.

Two Regions use a DynamoDB global table, which replicates each write to the other Region, usually within a second. Creates go to one home Region. The redirect service in the other Region does the same "retry before 404" trick, but against the home Region, for links that haven't replicated yet. For failover, CloudFront's origin group retries a failed request on the second Region's ALB. Two details trip people up. It only fails over GET, HEAD and OPTIONS requests, which is fine for redirects. And it only treats timeouts and refused connections as failures if you include 504 and 503 in its list of failover status codes. Its defaults are also slow: CloudFront makes 3 connection attempts to the failing origin, each with a 10-second timeout, before it fails over. A dead Region can hold a click for 30 seconds, and most people give up long before that. Set the primary origin's connection attempts and connection timeout to their minimums (1 attempt, 1 second), and keep its response timeout short, so failover costs about a second.

Argument 4: counting clicks the CDN served

In the generic design the redirect service published a click event. On AWS, about 70% of redirects never reach your service, because CloudFront answers them. So the click source moves to CloudFront itself.

%% caption: Clicks are counted from CloudFront's logs, so the 70% of redirects the edge answers are counted too.
flowchart LR
  CF["CloudFront<br/>real-time logs"] --> KDS[["Kinesis Data Streams"]]:::hot
  KDS --> CW["Count function<br/>Lambda"] --> CC[("DynamoDB<br/>click_counts")]:::store
  KDS --> FH["Firehose<br/>Parquet to S3"] --> ATH["Athena"]

CloudFront real-time logs send a log line for each request, with fields you choose (timestamp, path, country, referrer, status), to a Kinesis Data Streams stream within seconds. A Lambda function reads batches from the stream, adds up clicks per code, and writes one UpdateItem ... ADD clicks :n per code per batch. That's the same "aggregate, then one write per code" idea as before. Firehose reads the same stream and writes Parquet files to S3, partitioned by day, for Athena to query.

The costs are small next to the CDN: $0.01 per million log lines (about $100 a month), and a few hundred dollars for Kinesis in on-demand mode.

The production gotcha is in the consumer's defaults. A Lambda function reading Kinesis retries a failed batch forever, and records in a shard are processed in order. One malformed log line that crashes the function blocks its shard, so counts for every code in that shard silently stop. Set a maximum number of retries, turn on "split the batch in half on error" so the bad record gets isolated, send discarded records to an SQS queue, and alarm on the stream's iterator age. That alarm is what tells you counting has fallen behind.

Retries also mean a batch can be counted twice. For counts that only need to be "about right within a minute", that's acceptable. For counts you bill on, use the approach in the Ad clicks lab.

What it costs

For about 10 billion redirects a month, that's roughly $1.35 per million clicks. About three quarters of it is the CDN line, which is where a real team would spend its negotiating effort.

The same design on GCP and Azure

The design survives the move; only the names and limits change.

Whichever cloud you're on, ask the same questions: what each service charges per request, what one partition or key can take, and what its defaults do when something fails.