Invalidation and TTLs

There's an old joke among programmers that there are only two hard things in computer science: naming things, and cache invalidation. The joke lasts because it's true. Putting data in a cache is easy. Knowing exactly when a copy has stopped being true, and removing it at that moment from every place it lives, is where caching gets difficult.

For Tiffin, this is the difference between a fast menu and a customer ordering a dish that sold out twenty minutes ago. In this lesson we'll look at the tools for keeping caches honest: TTLs, explicit deletes, versioned keys and events. And at the one rule that makes stale data survivable when, eventually, they all fail.

Two ways a copy stops being trusted

Every cache entry needs a plan for when it stops being trusted. There are only two basic plans:

1. Time-based. The copy is trusted for a fixed time, then thrown away. That's a TTL.
2. Event-based. The copy is trusted until something changes, and the change removes it. That's explicit invalidation.

Most real systems use both: explicit invalidation for speed, a TTL as a backstop for when invalidation fails.

TTLs: staleness with a ceiling

A TTL is the simplest invalidation there is. Store the value with an expiry, and it disappears on its own:

SET nearby:v1:cell-8812 "<json>" EX 60

Nothing has to remember to delete it. No code on the write path needs to know the cache exists. And you get a hard guarantee: the data is never more than 60 seconds old.

The price is that it's always potentially up to 60 seconds old. Even if nothing changed, the entry expires and gets reloaded (wasted work). And if something changed one second after caching, readers see the old value for the other 59.

Choosing a TTL

Picking a TTL is a tradeoff between two numbers:

• How stale can this be before it hurts? Ask the product owner, not the engineers. "Can a customer see a closed restaurant for a minute?" is a business question.
• How much load does each expiry cause? Each expiry costs one database query. A 1-second TTL on a key read 1,000 times a second still saves 999 of those reads. A 1-second TTL on a key read once a minute saves nothing.

Some rough guides for Tiffin:

A useful habit: if you can't say what goes wrong when this data is stale for one TTL, you haven't finished choosing the TTL.

Explicit invalidation: delete on change

When a restaurant edits its menu, the server knows the menu just changed. So it deletes the key straight away (as we saw in the last lesson):

await db.markSoldOut(itemId)
await redis.del(`menu:v1:${restaurantId}`)

Now the menu is fresh on the very next request, not up to 10 minutes later. The TTL stays on the key, as a backstop for any delete that fails or never happens.

Explicit invalidation is precise, but it has a hidden cost that grows with your system: every place that changes the data must know every cache key that depends on it. Consider what depends on one dish's price:

• menu:v1:42, the restaurant's menu.
• menu:v1:42:hi, the same menu in Hindi.
• top-dishes:v1:week, if this dish is in the top 10.
• search:v1:biryani:cell-8812, if search results show prices.

Forget one, and that one stays stale until its TTL. Now picture the same dish being updated from the kitchen tablet, an admin tool, a bulk import script and a nightly sync with the restaurant's own point-of-sale system. Every one of those paths needs the same list of deletes. That's how invalidation bugs happen.

Two things help:

• Put all writes to a piece of data behind one function. updateMenuItem() is the only code that writes menu items, and it does the invalidation. Scripts and tools call it too, instead of running SQL directly.
• Use TTLs for derived data that's hard to track. Don't try to invalidate "top dishes" on every price change. Let it expire hourly.

Versioned keys: change the name instead

Sometimes deleting is awkward, for example when the copies are spread across many places you can't easily reach. Another option is to never delete anything, and instead change the key readers look for.

Keep a version number for each menu in the database. Every change bumps it:

UPDATE restaurants SET menu_version = menu_version + 1 WHERE id = 42;

Readers build the cache key from the current version:

menu:42:v17   ->   (menu changes)   ->   menu:42:v18

(This v17 counts changes to the data. It's a different thing from the v1 in menu:v1:42, which marks the format of the cached value.)

Old versions are never read again; their TTL cleans them up. This is the same idea as the versioned photo URLs from lesson 2, applied to data. It works especially well for caches you can't easily reach to delete from: phones, CDNs, and the in-memory caches of many servers. They all simply start asking for v18.

The catch: readers must look up the current version first. That's one more (cheap) read per request, often itself cached for a few seconds.

Events: let the database announce changes

In larger systems, a different approach is common: instead of every write path deleting keys, the database's own change log is turned into a stream of events ("menu item 7 changed"), and a separate process listens and invalidates the right keys. Because it reads changes from the database itself, it catches every write, including ones from scripts and tools that bypass your code.

This is called change data capture, and we'll meet it properly in Module 17. For Tiffin on one server, deleting in updateMenuItem() is plenty.

The race that's left

In the last lesson we saw that "update the database, then delete the key" avoids most races. One rare one remains:

The cache now holds the old menu, and nothing will remove it until the TTL. It needs a reader that's slower than an entire write, at exactly the wrong moment, so it's rare. But on a busy system, rare things happen every day.

Practical defences, from simplest:

• Keep the TTL as a backstop. The stale menu lives 10 minutes, not forever.
• Delete twice. Delete the key after the write, and again a second or so later. This catches most readers that were mid-flight. It's crude but common.
• Use versioned keys. The slow reader stores its old menu under v17, which nobody reads any more. (This works as long as the reader reads the version number and the menu together, in one query, so it never pairs an old menu with a new number.)

The rule that saves you

All of these make stale data rarer and shorter-lived. None makes it impossible. Caches spread copies across machines, and copies can always be briefly wrong.

So the most important rule in caching isn't about invalidation at all:

The cache decides what people see. The database decides what happens.

When a customer places an order, the server checks every dish's availability and price against the database, inside the order transaction, not against the cached menu. If a dish sold out after the menu was cached, the order is refused with a clear item_unavailable error, and the app shows "Sorry, the paneer tikka just sold out."

A stale menu is a small annoyance. A charged card for food that doesn't exist is a real problem. As long as every decision reads the source of truth, staleness stays an annoyance.