Server-sent events

Long polling gets rid of empty answers, but it still closes the connection after every update and opens a new one. What if the server could just keep the response open and write to it whenever there's news?

That's exactly what server-sent events (SSE) do. It's a standard built into every modern browser, it runs over ordinary HTTP, and if you've used a chat app that types its answer out word by word, you've probably watched it working.

One response that never ends

A normal HTTP response has a body, and then it ends. An SSE response starts like any other, but the body never finishes. The server keeps the connection open and writes a small chunk of text each time something happens.

Server-sent events are a single long-lived HTTP response that the server keeps writing messages into.

Tiffin's customer app is a native phone app, not a web page. This lesson still shows the browser's built-in EventSource, because it is the reference implementation of the standard and the shortest way to see how it works. Native apps use a small library that speaks the same format. Pick one that also does what EventSource does on its own, automatic reconnect and Last-Event-ID resume, or build those yourself; a client that only parses the stream loses every update sent while it was disconnected (more on native apps below).

The browser side is a few lines:

const events = new EventSource('/orders/123/events')

events.addEventListener('status', (e) => {
  const update = JSON.parse(e.data)
  render(update)
})

On the server, you answer with the content type text/event-stream, don't close the response, and write messages in a simple text format. Each message is a few field: value lines followed by a blank line:

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache

id: 7
event: status
data: {"status":"accepted","eta_minutes":35}

id: 8
event: status
data: {"status":"picked_up","rider":"Arjun"}

: keep-alive

• data is the message itself. It's text, so you usually send JSON.
• event is a name, so the client can listen for different kinds of messages.
• id is the cursor from the last lesson, built into the protocol.
• A line starting with : is a comment. The browser ignores it, which makes it useful as a heartbeat.

%% caption: One request, many messages, all flowing from the server to the client.
sequenceDiagram
  autonumber
  participant C as Customer app
  participant S as Server
  C->>S: GET /orders/123/events
  Note over S: keeps the response open
  S-->>C: event: accepted
  S-->>C: event: picked up
  S-->>C: : keep-alive
  S-->>C: event: delivered

Reconnecting is built in

This is SSE's best feature, and the reason to pick it over hand-rolled streaming.

When the connection drops (the customer walks into a lift, the load balancer restarts, your server is deployed), EventSource reconnects on its own after a short delay, usually around 3 seconds. The server can change that delay by sending a retry: 10000 line, which is handy for spreading out reconnects after a big outage.

Better still, when it reconnects it sends a header with the last ID it received:

GET /orders/123/events
Last-Event-ID: 8

Your server reads it and sends everything after 8 before streaming new messages. The cursor idea from long polling is now part of the standard, and you get gap-free reconnects without writing any client code. All you need on the server is a way to look up "events for this order after ID 8", which for Tiffin is just the order's status history table.

It's just HTTP, mostly

Because SSE is a normal HTTP response, it works with your existing setup: the same load balancer, the same cookies for login, the same TLS certificates. There are a few traps, though.

Buffering proxies

Some proxies collect a response into a buffer and send it on once they've got enough. For a normal page that's helpful. For a stream it's a disaster: your messages sit in nginx's buffer and the customer sees nothing. Turn buffering off for event routes. With nginx, set proxy_buffering off, or send the header X-Accel-Buffering: no from your app. Compression middleware can buffer too.

Idle timeouts

The same 60-second idle timeouts from the last lesson apply. If no message is sent for a minute, something in the path may cut the connection. Send a comment line like : keep-alive every 15 to 30 seconds. It costs a few bytes and keeps every proxy happy.

The six-connection limit

This one is about browsers, so it matters for Tiffin's website and the restaurant dashboard, not the native apps, which manage their own connections. Over HTTP/1.1, browsers allow about 6 connections per domain, shared across all tabs. Each open EventSource holds one of them forever. A customer with Tiffin open in 6 tabs has used them all, and the seventh tab's normal API calls just hang.

HTTP/2 fixes this by carrying many streams over one connection, typically 100 or more. If you serve SSE, make sure it goes over HTTP/2. Most load balancers and CDNs support it today.

No custom headers

The browser's EventSource can't set headers like Authorization: Bearer .... It sends cookies, so cookie-based login works out of the box. If you use tokens, you either pass a short-lived token in the URL or use a small library that builds SSE on top of fetch, which can set headers.

What SSE can't do

The name says it: server-sent. Messages only flow from the server to the client. If the client needs to send something, it uses an ordinary request, like POST /orders/123/cancel. For a lot of features, that's completely fine. Think about what a Tiffin customer actually does on the tracking screen: watches. They might tap "call rider" once. They don't send a stream of messages.

The other limits:

• Text only. SSE carries UTF-8 text. Binary data needs encoding (base64 adds about 33%), so it's not a good fit for audio or images.
• Native mobile apps don't get EventSource for free. Android and iOS need a small library or a few dozen lines of code to read the stream. It's still just an HTTP response, so it's not hard.

Server cost

Like long polling, SSE holds one open connection per watching client, so the server needs to be built for many idle connections, not a thread per connection. The difference is what happens per update:

For Tiffin's 10,000 tracking screens, SSE means 10,000 open connections and about one small message per screen every few minutes, plus a heartbeat every 20 seconds. That's around 500 heartbeats a second at a few bytes each, which is nothing.

Where this leaves Tiffin

SSE is a strong fit for the customer tracking screen: one-way, delivered through infrastructure Tiffin already has, with free reconnect-and-resume. It doesn't fit the rider app as neatly. The rider sends location updates every few seconds and receives delivery offers they need to accept. That's traffic in both directions, all the time. For that, we need a connection that carries messages both ways.