WebSockets

The Tiffin rider app is busy in both directions. Every 4 seconds it reports where the rider is. Every so often the server offers it a new delivery, and the rider has 20 seconds to accept. Then come pickup confirmations, "customer not answering" alerts, and chat messages from the support team.

You could build that from SSE for the server-to-rider direction plus a POST every 4 seconds for the rider-to-server direction. It would work. But it's two channels to manage for what is really one conversation. WebSockets give you a single connection where either side can send a message at any time.

From HTTP to something else

A WebSocket connection starts life as an HTTP request. The client asks the server to switch protocols:

GET /rider/socket HTTP/1.1
Host: api.tiffin.app
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13

If the server agrees, it answers with 101 Switching Protocols, and from that moment the TCP connection is no longer speaking HTTP. It carries WebSocket frames: small messages with a header of just 2 to 14 bytes, that either side can send whenever it likes.

A WebSocket is a long-lived connection, opened with an HTTP handshake, over which client and server can both send messages at any time.

%% caption: One connection, messages in both directions, no request and response pairs.
sequenceDiagram
  autonumber
  participant R as Rider app
  participant S as Server
  R->>S: GET /rider/socket, upgrade
  S-->>R: 101 Switching Protocols
  Note over R,S: socket open, both sides send
  R-)S: location
  R-)S: location, 4 s later
  S-)R: new delivery offer
  R-)S: location

In the browser the API is small:

const socket = new WebSocket('wss://api.tiffin.app/rider/socket')

socket.onmessage = (e) => {
  const msg = JSON.parse(e.data)
  if (msg.type === 'offer') showOffer(msg)
}

setInterval(() => {
  socket.send(JSON.stringify({ type: 'location', lat, lng }))
}, 4000)

Always use wss:// (WebSockets over TLS), not ws://. Besides the obvious security reason, many mobile networks and corporate proxies mangle or block unencrypted WebSocket traffic, while encrypted traffic passes straight through.

What you get

Two-way, low-latency messages. Either side sends whenever it wants. A delivery offer reaches the rider in one network trip, and their "accept" comes back the same way.

Tiny per-message cost. After the handshake, there are no HTTP headers per message. A Tiffin location update is about 60 bytes of JSON plus a few bytes of frame header. Sending the same thing as a POST would carry 500 to 1,000 bytes of headers each time.

Let's put a number on that. Tiffin has about 6,000 riders online at the dinner peak, each sending a location every 4 seconds:

6,000 riders / 4 seconds = 1,500 location messages per second

As POST requests:  1,500 x ~800 bytes  = ~1.2 MB/s, 1,500 requests/s through the app
As socket frames:  1,500 x ~70 bytes   = ~105 KB/s, no request handling overhead

Neither is huge, but the socket version does around a tenth of the work, and on a rider's phone, fewer bytes and fewer radio wake-ups mean better battery life on a 10-hour shift.

Binary if you want it. Frames can carry text or binary, so you can send compact encodings like Protocol Buffers if bandwidth matters.

What you have to build yourself

WebSockets give you a pipe and very little else. Compared with SSE, you have to handle several things on your own.

Reconnecting

There's no automatic reconnect. When the rider rides into a basement car park and the connection drops, your code has to notice, wait, and reconnect. Use exponential backoff with jitter (wait 1s, then 2s, then 4s, up to a cap like 30s, each with some randomness), so that thousands of riders reconnecting after an outage don't all arrive at once.

Resuming

There's no Last-Event-ID either. If a delivery offer was sent while the rider was disconnected, it's gone unless you design for it. The same cursor idea applies: the client sends the ID of the last message it processed when it reconnects, and the server replays anything newer. Or, simpler, the client asks for current state on reconnect ("what offers and deliveries do I have right now?") and carries on from there.

Knowing the connection is dead

A connection can die silently. The phone loses signal, and no packet ever arrives saying "goodbye". The server keeps a socket it thinks is open, and sends offers into the void. WebSockets have built-in ping and pong control frames for this. The server sends a ping every 20 to 30 seconds. If no pong comes back in time, it closes the connection and treats the rider as offline. Those regular pings also keep idle-timeout proxies from cutting the connection.

Message types

There's no event: field. Everything is just "a message", so you invent your own envelope, usually JSON with a type field, as in the code above.

Authentication

You check identity during the handshake, usually with a cookie or a token. The browser's WebSocket API can't set an Authorization header, so tokens usually travel as a short-lived ticket in the URL or as the first message. If you rely on cookies, check the Origin header during the handshake: otherwise any website the user visits can open a socket to your server with their cookies attached. Finally, a rider's shift is 10 hours, but their access token might expire after one. Decide what happens then: close the socket and let the app reconnect with a fresh token, or have the client send a new token as a message.

Running them in production

Long-lived connections change how the rest of your system behaves.

Load balancers must support the upgrade. Modern ones do, but check that idle timeouts are longer than your ping interval.

Deploys get harder. In Module 4, a rolling deploy drained each server by waiting for in-flight requests to finish, which takes seconds. A WebSocket never finishes. So you close sockets on purpose during a deploy, and every client on that server reconnects to another one. If a server holds 20,000 sockets, that's 20,000 reconnects within seconds. Close them gradually over a minute or so, rather than all at once.

Balance drifts. A load balancer only places new connections, and a socket stays where it landed for hours. After a rolling deploy, the servers restarted last sit nearly empty while the others are full. A server you add to cope with the dinner rush gets almost nothing, because existing riders never reconnect. Use a least-connections algorithm for socket traffic, and occasionally ask clients on overloaded servers to reconnect.

Capacity is about connections, not requests. An idle socket costs very little CPU but some memory, typically 10 to 50 KB per connection depending on your language and buffers. An event-loop server with a few GB of RAM can comfortably hold tens of thousands of connections, and well-tuned ones hold hundreds of thousands. Watch open file descriptors too: each socket is one, and the default operating system limit is often just 1,024.

Backgrounded apps. iOS and Android suspend apps in the background and their sockets with them. When the app is closed, no live-update technique in this module reaches it. That's what mobile push notifications (Apple's APNs and Google's FCM) are for: they go through a connection the operating system itself keeps open. Tiffin uses push for "your food has arrived" when the app is closed, and the socket when it's open.

Choosing between the four

A rule of thumb: if the client mostly listens, use SSE. If the client also talks a lot, use WebSockets. If you have few clients and can wait a few seconds, just poll.