Module 21 / Key technologies

Module 21 · lesson 4 of 7 · Read 8 min

CDNs

Edges, origins, push and pull, and cache keys.

What you'll learn

  1. The parts
  2. What a CDN is best at
  3. How the edge decides: Cache-Control
  4. Cache keys
  5. Pull or push
  6. Invalidation: version instead
  7. Tiffin's setup
  8. Limits and failure modes
  9. When not to use it

Tiffin's servers live in one region, say Mumbai. A customer in Mumbai loads the restaurant list in 40 ms. Tiffin launches in Singapore, and the same page takes 400 ms, because every image and every request crosses the sea, and opening a TCP and TLS connection costs two or three round trips before the first request even leaves.

A content delivery network (CDN) fixes this by keeping copies of your responses in hundreds of data centres close to users. Cloudflare, Amazon CloudFront, Fastly and Akamai are the big ones. Module 3 explained caching in general and Module 18 covered routing users globally. This lesson is about the CDN specifically: how it decides what to store, for how long, and how it goes wrong.

The parts

  • Edge (or point of presence, PoP): a CDN data centre near users. Large CDNs have hundreds of them.
  • Origin: your own server or S3 bucket that holds the real content.
  • Cache hit: the edge already has the response and answers directly.
  • Cache miss: the edge fetches from the origin, stores the result, then answers.

A CDN is a shared cache that sits in many cities between your users and your origin.

%% caption: Most requests stop at the nearby edge. Misses go through a mid-tier "shield" so the origin sees very few.
flowchart TD
  C(["Customer in Singapore"]) --> E["Edge in Singapore<br/>hit, about 15 ms"]:::hot
  E -. "miss only" .-> S["Origin shield in Mumbai"]
  S -. "rare" .-> O[("Origin")]:::store
Most requests stop at the nearby edge. Misses go through a mid-tier "shield" so the origin sees very few.

How does a user reach the nearest edge? Two common ways. With anycast (Cloudflare, Fastly), every edge announces the same IP address and internet routing sends packets to the closest one in network terms, which is usually, but not always, the closest geographically. With DNS-based routing (Akamai, and mostly CloudFront), the CDN's DNS answers with the address of an edge near the user's resolver (the DNS lesson in Module 22 covers this).