Stateless servers

Tiffin adds a second server on a Monday. By Tuesday, support has three new kinds of complaint:

• "The app keeps logging me out, sometimes twice in one order."
• "Half our dish photos are broken," from a restaurant that just updated its menu.
• "My cart emptied itself when I went to checkout."

Nothing crashed. No error was logged. The code that worked perfectly on one server is now wrong about half the time. All three bugs have the same cause: each server was keeping something in its own memory or on its own disk, and the next request went to the other server.

What stateless means

Every app has state: who's logged in, what's in each cart, every order ever placed. The question is where it lives.

A stateless server keeps nothing between requests that another server would need. Any server can handle any request, because everything that must outlive one request lives in a shared place.

"Stateless" doesn't mean the app has no state. It means the servers don't hold it. The state moves to places built to hold it and share it: the database, a shared cache like Redis, and object storage for files.

%% caption: Stateless servers. Any of them can answer, because the state lives in shared stores that all of them use.
flowchart TD
  LB["Load balancer"] --> S1["Server 1"] & S2["Server 2"] & S3["Server 3"]
  subgraph shared ["Shared state"]
    DB[("Database<br/>orders, carts")]:::store
    R[("Redis<br/>sessions")]:::store
    OS[("Object storage<br/>photos")]:::store
  end
  S1 & S2 & S3 --> shared

Once the servers are stateless, they're disposable. You can add one, remove one, kill one mid-dinner, or replace all of them with a new version, and no customer loses anything except maybe a request that was in progress.

Let's fix Tiffin's three bugs.

Bug 1: sessions in memory

When a customer logs in, Tiffin's server creates a random session ID, stores { customer_id: 812 } under that ID in a dictionary in memory, and sends the ID back as a cookie. On each later request, the server looks up the cookie in its dictionary.

With one server, that works. With two, the login lands on server 1, and the next request has a 50% chance of landing on server 2, whose dictionary is empty. Logged out. There are three common fixes.

%% caption: The login landed on server 1. The next request landed on server 2, which has never heard of this session.
sequenceDiagram
  autonumber
  participant C as Customer app
  participant L as Load balancer
  participant S1 as Server 1
  participant S2 as Server 2
  C->>L: log in
  L->>S1: log in
  Note over S1: stores the session in its own memory
  S1-->>C: cookie with the session ID
  C->>L: next request, with the cookie
  L->>S2: next request
  Note over S2: empty dictionary, unknown session
  S2-->>C: logged out

Fix A: sticky sessions

Tell the load balancer to always send the same customer to the same server, usually using a cookie it sets itself. The bug disappears instantly, with no code change.

It's a crutch, though, and it breaks in exactly the moments you care about:

• When server 1 dies or is removed for a deploy, everyone stuck to it is logged out.
• When autoscaling removes servers after dinner, their customers are logged out.
• Load becomes uneven. A server that got a few heavy users early stays busy, and the balancer can't move them.

Sticky sessions are fine as a stopgap for a week. They're not a design.

Fix B: a shared session store

Keep the session lookup exactly as it is, but put the dictionary in Redis instead of server memory:

key:    session:9f3ac1e0...
value:  { "customer_id": 812, "city": "blr", "created": "2026-10-12T19:02:11Z" }
expiry: 30 days

Every server reads the same Redis, so every server knows every session. A Redis lookup on the same network takes well under a millisecond, which is nothing next to a 50 ms request.

Logging someone out is easy: delete the key. "Log out of all devices" is easy too, if you also keep a list of each customer's sessions.

The cost is a new dependency. If Redis is down, nobody's logged in. So Redis needs a replica and failover, just like the database (Module 5 covers how that works).

Fix C: signed tokens

Instead of storing the session anywhere, put the facts in the cookie itself and sign it with a secret key that only the servers know. A common format is the JSON Web Token (JWT):

payload:   { "customer_id": 812, "exp": 1791832631 }   (exp = expiry time, in seconds since 1970)
signature: HMAC-SHA256(header + payload, server_secret)

On each request, the server checks the signature. If it's valid, it trusts the payload, with no lookup at all. Any server with the secret can do this.

The catch is revocation. A token is valid until it expires, and there's no central place to cancel it. If a customer's phone is stolen, you can't easily log out that token. The usual answer is to make tokens short-lived (say 15 minutes) and pair them with a longer-lived refresh token that is checked against the database when it's used to get a new short token. You end up with some server-side state anyway, just touched far less often.

For Tiffin, a shared session store in Redis is the simplest robust choice. Tiffin already runs Redis for caching (Module 3), and being able to log out a stolen phone instantly matters for an app that stores payment methods.

Bug 2: files on local disk

When a restaurant uploads a new dish photo, the server saves it to /var/tiffin/uploads/dish_4411.jpg and stores that path in the database. A customer's request for the image goes to the other server, which doesn't have the file. Broken image.

Copying files between servers is a trap. You'll spend months on sync scripts that miss files.

The real fix is to stop treating the server's disk as storage. Put files in object storage, like Amazon S3, Google Cloud Storage or Azure Blob Storage. These are built for exactly this: unlimited files, very high durability, accessible from any server, and cheap. Store only the object's key in the database:

UPDATE menu_items
SET photo_key = 'dishes/4411/9b2e.jpg'
WHERE id = 4411;

Then serve the images through a CDN (Module 3), so customers download them from a nearby edge location rather than your servers at all.

You can go one step further. Instead of the kitchen tablet uploading a 4 MB photo through your app server, the server hands it a presigned URL: a short-lived link that allows one upload, directly to object storage. The photo never passes through your servers, which saves bandwidth and keeps slow uploads from tying up workers.

There's a second reason to keep files off server disks. With autoscaling, servers are created and destroyed every day. Anything written to their disks disappears with them.

Bug 3: the cart in memory

To save database writes, an early version of Tiffin kept each customer's cart in a dictionary in server memory. Same bug: add a dish on server 1, open the cart on server 2, and it's empty.

The cart needs to live somewhere shared. The options:

• The database. A carts table with one row per item. Durable, easy to query, and you already have it. One more write per tap.
• Redis, with an expiry of a day or two. Fast, and abandoned carts clean themselves up. Fine if losing a cart in a rare Redis failure is acceptable.
• The client. The app keeps the cart on the phone and sends it in full at checkout. The server holds nothing. But the cart can't follow the customer from phone to laptop.

Tiffin puts carts in Redis with a 48-hour expiry: they're short-lived, written often, and losing one is annoying but not costly. Orders, which must never be lost, go to the database.

Other state hiding in your servers

Sessions, files and carts are the obvious ones. The horizontal scaling lesson listed the rest: in-memory caches, scheduled jobs, rate-limit counters, open WebSocket connections and local log files. Each has its own fix in a later module. Add one more to that list: an in-memory counter for order numbers, which lets two servers hand out the same number (Module 19 covers unique IDs).

Some of these, like WebSockets, can't be made fully stateless: a connection really is attached to one server. The goal then is to make losing that server cheap, so the client reconnects to another one and carries on.

The test

Here's a simple question to ask about any server in your system:

"If I terminate this server at a random moment during dinner, what does any user lose?"

The right answer is "at most the requests it was handling at that moment, and the clients retry those". If the answer includes "their login", "their cart" or "the photos uploaded today", something stateful is still hiding on the server.

Netflix made a habit of testing this for real, with a tool called Chaos Monkey that kills random production servers on purpose.