WhatsApp: Chat

Sending a chat message sounds like the simplest thing a server can do: take some text from one phone and hand it to another. The catch is the other phone. It's in a pocket, on a train, in a lift, switched off, or on a network that drops every few minutes. And the person who sent the message is staring at a tick, waiting for it to turn into two.

In this lab you'll design a chat system modelled on WhatsApp. The hard part isn't sending. It's delivering, exactly once as far as the user can tell, in the right order, to phones that are often not there.

The brief

People send text messages to each other, one to one and in groups. Each message shows its status with ticks:

• One tick: the server has it.
• Two ticks: it reached the recipient's phone.
• Blue ticks: the recipient read it.

Phones go offline all the time. When a phone comes back, it should receive everything it missed, in order, with nothing missing and nothing shown twice.

In scope:

• One-to-one messages and small groups (up to 1,000 members).
• Sent, delivered and read receipts.
• Offline delivery and push notifications.
• Online status and "last seen", briefly.

Out of scope: voice and video calls, the details of end-to-end encryption (we'll assume the server only ever sees encrypted blobs), media transcoding, backups, and contact discovery. Media is covered only as "upload once, send a link".

Requirements

Functional

• A user can send a text message to another user or to a group.
• The recipient gets it in real time if online, or when they next connect if offline.
• The sender sees sent, delivered and read status per message (per member, for groups).
• Messages in a conversation appear in the same order for everyone in it.
• An offline user gets a push notification for new messages.

Non-functional

• Low latency: when both phones are online, a message arrives in under 500 ms at p99, anywhere in the world.
• Never lost: once the sender sees one tick, the message will be delivered, even if servers crash.
• No visible duplicates: a retry must never show the same message twice.
• Ordered within a conversation. Order across different conversations doesn't matter.
• Battery and data friendly: phones must not poll.
• The server doesn't keep history. Once a message is delivered to every recipient device, the server deletes it. History lives on phones.

That last one is a product choice that changes the design a lot. The server is a delivery system, not an archive.

Numbers

Assumptions, for a very large chat app:

• 2 billion users, 1 billion daily active.
• 100 billion messages a day.
• Average message: ~100 bytes of text, ~300 bytes with encryption overhead and metadata.

Message rate:

• 100B / 86,400 s ≈ 1.2 million messages/s on average.
• Peak (New Year's Eve, festivals) at 3× or more: ~4 million/s.
• Each message also produces a delivered receipt and usually a read receipt. So ~3 events per message, and the real event rate is ~3.5M/s average, 10M+/s at peak.

Connections:

• Say 50% of daily users are connected at the busiest hour: 500 million open connections.
• A well-tuned server holding mostly idle connections can manage around 1 million. You need ~500 connection servers, more with headroom. Plan for 1,000, so each holds about 500,000 at the busiest hour: half of what it could, which leaves room for a dead server's phones to reconnect elsewhere.

Storage on the server:

• Most messages are delivered within seconds and deleted. Say 10% wait for an offline phone, for an average of a day.
• Pending at any moment: 100B × 10% × 300 B ≈ 3 TB. Tiny.
• If you also keep a 30-day undelivered limit for long-offline phones, budget a few tens of TB. Still small.

Bandwidth:

• 4M messages/s × 300 B × 2 (in and out) ≈ 2.4 GB/s at peak, plus receipts. Modest. Media goes through a separate blob store and CDN.

The numbers say this is not a storage problem, and not really a bandwidth problem. It's a connections and delivery guarantees problem: half a billion long-lived connections, and millions of tiny events a second that must each land exactly once.

API

Clients keep one persistent connection (WebSocket or a custom protocol over TCP) and exchange small frames. There's barely any REST.

client -> server   SEND     { client_msg_id: "c-7f3a", chat_id: "u:42|u:97", body: <encrypted> }
server -> client   ACK      { client_msg_id: "c-7f3a", msg_id: 99812, seq: 1408, ts: ... }   (one tick)

server -> client   MESSAGE  { msg_id: 99812, chat_id: ..., seq: 1408, from: 42, body: <encrypted> }
client -> server   DELIVERED { msg_ids: [99812] }                                            (two ticks)
client -> server   READ      { chat_id: ..., up_to_seq: 1408 }                                (blue ticks)

server -> client   RECEIPT  { msg_id: 99812, status: "delivered", by: 97 }

client -> server   SYNC     { last_acked: { "<chat_id>": 1403, ... } }   (on reconnect)

Things to notice:

• A one-to-one chat's chat_id is the two user IDs in a fixed order (u:42|u:97), so both phones work out the same ID without asking the server. A group's chat_id is its group ID.
• The client makes up client_msg_id before sending. That's what lets the server spot retries (more below).
• The server assigns seq, a per-chat sequence number. That's what gives order.
• Read receipts say "read up to seq 1408", not "read message 99812", so one frame covers many messages.

Data model

The server stores four things.

sessions            user_id + device_id  ->  connection server, connected_at     (Redis, in memory)

inbox               (user_id, device_id) ->  ordered list of pending messages     (wide-column store)
                    row key: user+device,  column key: msg_id
                    value: chat_id, seq, from, encrypted body, created_at

chat_sequence       chat_id  ->  last seq                                         (wide-column store)

group_members       group_id ->  list of user_ids                                (wide-column store)

Why a wide-column store like Cassandra or HBase for the inbox?

• Writes dominate: every message is written once and deleted soon after.
• Access is always "everything pending for this user's device, in order", which is a single-partition range read when the row key is the device and the column key is ordered.
• It spreads across hundreds of machines by user ID with no central bottleneck.

One caveat worth knowing. In Cassandra-style stores a delete doesn't remove anything at once. It writes a tombstone, a marker that says "this was deleted", which stays around so that a replica that missed the delete can't bring the data back. A table where nearly every row is deleted soon after it's written is a well-known way to make reads slow, because each range read has to step over the tombstones until compaction clears them. Queue-like tables like this one need care: a short gc_grace (how long tombstones are kept before compaction may drop them, 10 days by default in Cassandra) and compaction tuned for deletes, or a store whose deletes are cheap.

The session table is in memory because it changes constantly (every connect and disconnect) and is read on every message. If it's lost, phones simply reconnect and re-register.

High-level design

%% caption: The chat service is the hub. It stores each message, finds the recipient's connection server, and falls back to a push notification. Media skips the message path.
flowchart TD
  PA([Phone A]) --> CS1["Connection servers<br/>hold sockets"]:::hot
  PA -- media --> MS[("Media store<br/>blobs and CDN")]:::store
  CS1 --> CH["Chat service<br/>route, sequence, store"]:::hot
  CH --> SR[("Session registry<br/>who is where")]:::store
  CH --> IB[("Inbox store<br/>pending per device")]:::hot
  GS[Group service] -- "group members" --> CH
  CH -- "B online" --> CS2["B's connection server"] --> PB([Phone B])
  CH -. "B offline" .-> PS["Push service<br/>APNs, FCM"] -.-> PB

Each box, traced back:

• Connection servers exist because phones must not poll (battery) and messages must arrive in under 500 ms. They hold the persistent sockets and nothing else. Keeping them dumb means they can be restarted without losing data.
• Chat service assigns sequence numbers and decides where each message goes. It's stateless.
• Session registry answers "which connection server is user 97's phone on right now?"
• Inbox store meets "never lost": a message is written here before the sender gets one tick.
• Push service wakes offline phones through Apple's and Google's push systems.
• Group service turns one group message into one delivery per member.
• Media store keeps big files out of the message path.

Deep dive: never lost, never doubled

Follow one message from A to B.

1. A's phone saves the message locally with a fresh client_msg_id, shows a clock icon, and sends SEND.
2. The connection server passes it to the chat service.
3. The chat service checks whether it has already seen (sender, client_msg_id). If yes, it just re-sends the earlier ACK. If no, it assigns a seq for the chat, writes the message to B's inbox, records the client_msg_id, and then sends ACK to A. A shows one tick.
4. The chat service looks up B in the session registry. If B is online, it sends MESSAGE to B's connection server, which pushes it down B's socket.
5. B's phone stores it, shows it, and replies DELIVERED.
6. The chat service deletes it from B's inbox and sends a RECEIPT to A. A shows two ticks.

Where it can fail

A's network drops after sending, before the ACK. A doesn't know whether the server got it. A retries with the same client_msg_id. The server recognises it in step 3 and doesn't store it twice. That dedupe record only has to cover the usual retry window, minutes. A phone that retries much later, after days offline, can slip past it, and then B's phone catches the copy instead (see the exercise below).

The server crashes after writing to the inbox, before pushing to B. The message is safe in the inbox. When B next syncs, or when a sweeper notices undelivered messages for an online user, it gets sent.

B receives it, but the DELIVERED frame is lost. The server still has it in the inbox and sends it again later. B's phone sees a msg_id it already has and ignores it, but still replies DELIVERED.

So the network delivers at least once, and both ends deduplicate by ID. The user sees exactly once. Module 12 covers why you can't get exactly-once from the network itself, and Module 15 covers delivery guarantees.

The rule that makes it all work: the ACK to the sender is sent only after the message is durably stored. One tick is a promise, so you only make it once you can keep it.

Deep dive: order in a conversation

Two people in a chat send messages at almost the same time. Their phones' clocks disagree by a few seconds (Module 12: clocks lie). If you order by the sender's timestamp, the two phones may show different orders, and a reply can appear above the message it answers.

Instead, the server gives each message a sequence number per chat: 1405, 1406, 1407. Every member's phone sorts by seq. Everyone sees the same order, because there's one source of truth.

How do you hand out sequence numbers without a bottleneck?

• Each chat's counter lives in exactly one partition, chosen by hashing chat_id. All messages for one chat go through the same partition owner, which increments the counter. Module 15 covers this same "order within a partition, choose the key" idea in Kafka.
• Different chats go to different partitions, so there's no global counter, and the system scales with the number of chats.
• A one-to-one chat gets a few messages a second at most. Even a busy 1,000-member group rarely exceeds a few hundred. One partition owner handles that easily.

Sequence numbers also make gaps visible. If B's phone has 1405 and then receives 1408, it knows 1406 and 1407 are missing and asks for them, instead of silently showing an incomplete chat. The server can answer without keeping history: anything not yet delivered to B's device is still in B's inbox, so if 1406 and 1407 were only overtaken on the way, it sends them from there. A gap can also be permanent. The partition owner can crash after taking a seq and before the inbox write, and the sender's retry then gets a new seq, so 1406 never exists. So when B's inbox has nothing for a missing seq, and (for a group) the group log below has no unfinished fan-out holding it, the server says so, and the phone stops waiting for it instead of asking forever. If a "skipped" message does turn up later after all, the phone still accepts it and sorts it into place by seq.

What about a message from A to B while A is offline? A's phone shows it immediately, at the bottom, marked pending. When it's sent, the server gives it a real seq, and if other messages arrived in the meantime, A's phone re-sorts. This is why chat apps sometimes visibly move a message after reconnecting.

Deep dive: phones that are offline

B's phone is off for six hours. Messages pile up in B's inbox. Then B switches it on.

Waking the phone

When a message lands for B and the session registry says B is not connected, the chat service sends a push notification through Apple's or Google's push service. The notification is small: "new message from A", or even just "you have new messages", because the content is end-to-end encrypted. The phone wakes, opens a connection, and syncs.

Don't send one push per message to an offline user. If A sends 30 messages in a minute, collapse them: send one, then hold further pushes for that user for a short while.

Syncing on reconnect

%% caption: A message to a phone that's off. The inbox holds it, a push wakes the phone, and SYNC delivers it.
sequenceDiagram
  autonumber
  participant A as Phone A
  participant C as Chat service
  participant I as B's inbox
  participant P as Push service
  participant B as Phone B
  A->>C: SEND, client_msg_id
  C->>I: store, seq 41
  C-->>A: ACK, one tick
  Note over C: registry: B not connected
  C->>P: "you have new messages"
  P->>B: wake up
  B->>C: SYNC, last seq 40
  C->>I: read pending
  I-->>C: seq 41
  C->>B: MESSAGE seq 41
  B-->>C: DELIVERED
  C->>I: delete seq 41
  C->>A: RECEIPT, two ticks

B's phone connects and sends SYNC with the last seq it has for each chat. The server reads B's inbox (one partition range read), sends everything pending in seq order per chat, in pages of a few hundred, and B acknowledges each page. Each acked page is deleted from the inbox.

This works without server-side history because of one rule: a message leaves B's inbox only after B's device has acknowledged it. So everything B is missing is still in the inbox. The last seq per chat lets the server skip entries B already has but whose acknowledgement was lost.

Many devices

A user can have a phone, a laptop and a tablet. Each device gets its own inbox, keyed by (user_id, device_id). A message is deleted from one device's inbox when that device acks. "Delivered" (two ticks) is shown to the sender when the first of the recipient's devices acks, or when all have, depending on the product rule you choose.

Deep dive: online and last seen

The brief puts "online" and "last seen" in scope, briefly. The data mostly exists already: the session registry knows who is connected right now.

• Online means the user has a device in the session registry.
• Last seen is when the user's last device disconnected, or sent its last heartbeat if the connection died silently. Write it to a small key-value table, one write per disconnect, so it survives a registry restart.

The trap is telling other people. A user has hundreds of contacts, and phones connect and disconnect all day (every tunnel, every time the app goes to the background). Pushing every change to every contact would be hundreds of frames per connect or disconnect, across 500 million phones: more traffic than the messages themselves. So push only to people who are looking. When B opens the chat with A, B's phone subscribes to A's presence. It gets the current state once, then changes only while that chat stays on screen, and the subscription ends when B leaves the chat. The list of watchers lives next to A's entry in the session registry, and it's usually empty or one or two phones long. Also wait a few seconds before announcing "offline", so a quick reconnect doesn't send two updates.

Under load: what breaks

1. A connection server dies. A million phones lose their sockets at once and all reconnect within seconds. That's a thundering herd onto the other servers, the session registry and the inbox store (a million syncs). Fixes: clients reconnect with exponential backoff plus random jitter (Module 12), the load balancer spreads reconnects across many servers, and sync reads are paged so one user with 10,000 pending messages doesn't hog a server. Connection servers hold no data, so nothing is lost.

2. Stale session registry entries. A phone's connection silently dies (it went into a tunnel) and the registry still points at the old server for a while. Messages are pushed into a dead socket. Because they're still in the inbox until acked, nothing is lost. Add heartbeats every 30-60 seconds so dead sockets are noticed, and when a push isn't acked within a few seconds, treat the user as offline and send a push notification.

3. New Year's Eve at midnight. Message rate jumps 5-10× for a few minutes in each time zone, and most are to large groups. The inbox store's write rate spikes. Fixes: plan capacity for the known peak (this one is on the calendar), autoscale chat servers ahead of time, and let receipts degrade first. Delivering messages matters more than updating ticks, so receipt processing can lag by a minute and catch up.

4. A hot group. A 1,000-member group of an excited fan club sends 50 messages a second. Each is 1,000 inbox writes: 50,000 writes a second for one group, all sequenced by one partition owner. That owner is fine at assigning 50 sequence numbers, and the fan-out writes go to 1,000 different inbox partitions, so they spread out. The member cap is doing its job. If you ever allow 100,000-member "channels", that's a different design: fan-out on read, like a big account in a social feed.

5. A data centre goes down. Users connected there reconnect to another region. Their inboxes need to be readable there too, so the inbox store replicates across regions. Use a write consistency that waits for a local quorum (a majority of the replicas in this region) before the ACK, so one tick still means durable, and replicate to other regions asynchronously. Module 13 covers leaderless quorums, and Module 18 covers running in many regions.

Check against the requirements

Build it on AWS

A chat system at this size breaks the usual cloud advice. "Use the managed service" works when the bill is per request and requests are few. Here there are half a billion open connections and over a million messages a second, so anything priced per connection-minute or per message turns into the biggest number on the page. The interesting decisions are about which parts you can afford to rent and which you have to run yourself. Prices are us-east-1 list prices as of late 2026; treat them as rough and check the AWS pricing calculator before quoting them.

The architecture

%% caption: The chat design on AWS. Phones hold one TCP connection through an NLB; everything that must survive a crash is in DynamoDB.
flowchart TD
  PH([Phones]) --> NLB["Network Load Balancer<br/>TCP, idle timeout 350 s"]
  NLB --> CF["Connection fleet<br/>EC2, ~500,000 sockets each"]:::hot
  CF --> CH["Chat service<br/>ECS, stateless"]
  CH --> EC[("ElastiCache Valkey<br/>sessions and dedupe")]:::store
  CH --> IB[("DynamoDB inbox<br/>user#device, msg_id")]:::hot
  CH -. "phone offline" .-> PS["Push service<br/>calls APNs and FCM"]
  PH -- "media" --> S3[("S3 and CloudFront")]:::store

Argument 1: API Gateway WebSockets, or your own connection fleet?

API Gateway has a WebSocket mode that holds connections for you and calls your backend per message. No servers, no socket tuning. Price it at this size:

Connection minutes: 500 million phones x 43,200 minutes a month
                    = 21.6 trillion minutes x $0.25 per million       = ~$5.4 million
Messages:           each chat message is ~7 frames
                    (SEND, ACK, MESSAGE, DELIVERED, READ, 2 RECEIPTs)
                    100 billion a day x 7 x 30 = 21 trillion frames
                    x $0.80 per million (above the first billion)     = ~$16.8 million
                                                                     ~ $22 million a month

The limits rule it out before the bill does. An API Gateway WebSocket connection lasts at most 2 hours, and that can't be raised. Half a billion connections that each must reconnect every 2 hours is about 69,000 reconnects a second, all day. The default quota for new WebSocket connections is 500 a second per account per Region. It can be raised, but AWS's own documentation notes that at the default rate the 2-hour cap works out to about 3.6 million concurrent connections at most.

Now the fleet the lab already sized: about 1,000 connection servers, each holding around 500,000 mostly idle sockets. At Fargate's rates for 8 vCPUs and 32 GB, that's about $340 a server a month, or ~$340,000 for the fleet. That's an upper bound, because a connection tier like this usually runs on EC2 instances you tune yourself (file-descriptor limits, kernel socket buffers), and EC2 with a savings plan costs less. Add the Network Load Balancers. An NLB capacity unit covers 100,000 active TCP connections, so 500 million connections is about 5,000 units at $0.006 an hour, roughly $25,000 a month.

That's about $365,000 against $22 million: sixty times cheaper to run the sockets yourself. API Gateway WebSockets is a good tool for an app with tens of thousands of connections, where its per-message pricing is small and its quotas never bite. It isn't built for this.

The setting that matters on the NLB is its idle timeout. For TCP it defaults to 350 seconds and can be raised to 6,000; for a TLS listener it's fixed at 350. If neither side sends anything for longer, the NLB stops tracking the connection, and the next packet gets a TCP reset. So the 30-60 second heartbeat in this lab isn't only for noticing dead phones. It's also what keeps a quiet chat's connection alive through the load balancer.

Argument 2: what the inbox costs on DynamoDB

The inbox maps naturally onto DynamoDB. The partition key is user_id#device_id and the sort key is msg_id, so "everything pending for this device, in order" is one Query. A 300-byte message is one write unit to store and one to delete after the ack.

That's the catch. A queue-shaped table pays for every write and every delete, and this one does both a hundred billion times a day:

1 write + 1 delete per message = 2 write units
100 billion a day x 2 x 30 days = 6 trillion write units a month

On-demand:   6 trillion x $0.625 per million            = ~$3.75 million a month
Provisioned: ~3.5 million WCU (average 2.3 million, plus
             headroom for the evening peak)
             x $0.00065 per WCU-hour x 730 hours        = ~$1.7 million a month

That's for one inbox copy per message. Groups and extra devices multiply it. Three decisions shape this line:

• Provisioned, with reserved capacity. Traffic follows the day very predictably, so on-demand's flexibility isn't worth twice the price. Reserved capacity takes up to 77% off provisioned rates on a three-year term, which brings the inbox to roughly $0.4 million. New Year's Eve is on the calendar, so you raise capacity before midnight instead of hoping autoscaling keeps up.
• Keep the dedupe record out of DynamoDB. The lab's (sender, client_msg_id) check needs another conditional write per message, another ~$1.9 million a month on demand. But it only has to catch retries, and retries arrive within seconds or minutes. A key with a one-hour expiry in ElastiCache does the job: about 4 billion keys of ~60 bytes, a few hundred GB of memory. Keep the safe order from the "Which write comes first?" exercise: look the key up, write the inbox, and only then SET the key. If the cache is lost, the key has expired, or two copies of one retry race each other, a message is stored twice, and B's phone drops the copy by (sender, client_msg_id). As the lab says, duplicates can be cleaned up; lost messages can't.
• Know the per-partition ceiling. A partition accepts at most 1,000 write units a second. One device's inbox never comes close, and a 1,000-member group's fan-out spreads over 1,000 different partition keys. The design is safe here because the key is the device, not the chat. A table keyed by chat_id would melt on the first busy group.

For comparison, the lab's original choice, a self-run wide-column store like Cassandra or ScyllaDB on EC2 instances with local NVMe disks, has no per-write price at all. You pay for machines and for the people who run them, including the tombstone tuning the data model section warned about. At this volume many teams run it themselves. At a thousandth of this volume, DynamoDB on-demand is the obvious answer.

Argument 3: push through SNS, or straight to Apple and Google?

Amazon SNS can send mobile push notifications, and it handles tokens and both platforms behind one API. It charges per notification delivered. Apple's and Google's push services (APNs and FCM) charge nothing to send. With 10% of 100 billion daily messages waiting for an offline phone, even after collapsing pushes per user you send billions of notifications a day. At that count, a small push service of your own that keeps HTTP/2 connections open to APNs and FCM is cheaper than any per-notification price, and it lets you collapse and rate-limit pushes exactly the way the lab describes. For an app sending a few million pushes a day, SNS is the simpler choice.

Argument 4: should every inbox be in every Region?

DynamoDB global tables replicate every write to the other Regions, usually within a second. That sounds like exactly what failure 5 asks for. But every replicated write is billed again in each Region, so a global table across three Regions roughly triples the biggest line on the bill, and it does this for messages that are deleted seconds later.

The cheaper shape is a home Region per user plus one partner Region. A phone connects to its home Region. Its inbox is a global table replicated to exactly one partner Region, not all of them. If the home Region fails, phones reconnect to the partner and sync from the replica. Replication is asynchronous, so the last second of messages before the failure may not have arrived yet. The sender's phone never got its tick for anything that wasn't stored, so it resends those, and the gap closes. Expect the inbox line to roughly double rather than triple.

What it costs

That's about $0.70 per million messages, or about $0.25 with three-year reserved capacity on the inbox. The partner-Region replica adds roughly the inbox line again. Data transfer to the internet is extra, and at about a petabyte a month it's worth its own line in a real estimate. Around 80% of the money is DynamoDB writes and deletes, which is why the inbox design gets the most attention.

The same design on GCP and Azure

The design doesn't change. On every cloud, the connection tier is servers you run, and the inbox is a wide-column or key-value store priced per write, where provisioned capacity and keeping non-durable state out of it are the big levers.