Invalidation and Cache Problems

Why interviewers ask about this

Picture a café with a chalkboard that says "Today's menu." The cook ran out of pancakes but nobody wiped the board. Guests order pancakes, the waiter goes to the kitchen and comes back apologizing. A cache is exactly that kind of board: a fast copy of the truth. The main question isn't "how do we put it up" but "who wipes it, and when?"

There's a well-known joke: there are only two hard things in computer science, cache invalidation and naming things. Almost everyone proposes a cache in an interview. What sets a strong candidate apart is that they say on their own how the cache will go stale and break.

Why invalidation is the hardest part

The moment you add a cache, you have two sources of truth: the database and the cache. You can't update both at once, atomically; they're different systems. There's always a window between the two writes, and races slip into it.

The classic race with the "read from DB → put in cache" approach:

The three main ways to invalidate:

Approach	How it works	Pro	Con
TTL	the key expires on its own after N seconds	simple, self-healing	up to N seconds of stale data
Delete on write	after the UPDATE in the DB, we run DEL key	fresh almost immediately	races, forgotten write paths
Version in the key	user:42:v7, the version goes up on every change	old data simply stops being read	you need somewhere to store the version

💡 The rule: on a write, delete the key instead of updating it. If two writers update the cache, the order can get mixed up. After a delete, the next reader fetches fresh data from the DB. And always add a TTL as a safety net: even if a delete gets lost, the error lives for a limited time.

Stale data: how to live with it

Perfect freshness doesn't exist. Decide for each type of data separately how much staleness is acceptable:

• a news feed, a like counter: minutes, nobody cares;
• a user profile: seconds, plus delete on write;
• an account balance, stock levels at checkout: don't cache, or check against the DB at transaction time.

A useful technique is read-your-writes. A user who just changed something should see it right away. For that user, you can bypass the cache for a few seconds after the write. Everyone else can wait out the TTL.

Cache stampede: the avalanche on expiry

A popular key expires, and in that same millisecond thousands of requests get a miss and all go to the database to recompute the same thing.

Let's size it:

Parameter	Value
Load on the key	10,000 RPS
Recompute time in the DB	200 ms
DB queries during the recompute window	10,000 × 0.2 = 2,000 identical queries

The database falls over, requests pile up, timeouts, retries, and it gets even worse.

Three defenses

1. Locking (single-flight). Only the first request recomputes. It takes a lock with SET lock:key 1 NX EX 5, and the rest wait or get the old value. One query goes to the DB instead of 2,000. The price: extra latency for the waiters, and the risk that the lock holder dies. That's why the lock must always have a TTL.

2. Early recomputation. Refresh the key before it expires. For example, store a "soft" expiry next to it. Once that passes, one request refreshes the value in the background while everyone else keeps getting the current one (stale-while-revalidate). There's also a probabilistic version: the closer you get to the end of the TTL, the higher the chance that the next request triggers a refresh. The price: extra recomputes and more complex logic.

3. TTL jitter. If you warmed up 100,000 keys with the same TTL=300, in 5 minutes they'll all expire at once. Add randomness, for example TTL = 300 + random(0, 60), and the expirations spread out over a minute. It's cheap, nearly free, so jitter is always worth mentioning.

Hot keys

Even without expiry, a single key (a celebrity's post, the deal of the day) can get so much traffic that one Redis shard hits its CPU or network limit while the other shards sit idle. Sharding doesn't help here: the key still lives on a single node.

How to deal with it:

• a local in-memory cache in each API instance with a short TTL absorbs most of the requests;
• copies of the key key#0..N live on different shards, and reads go to a random copy (the downside: you have to invalidate all N copies);
• read replicas in Redis;
• detection: monitor the top keys so you find hot ones ahead of time.

How it sounds in an interview

When you add a cache, cover five points:

1. What we cache and why: "profiles, read:write ≈ 100:1, takes 90% of the load off the DB."
2. Strategy: cache-aside, DEL on write, plus a TTL as a safety net.
3. Acceptable staleness: "up to 60 seconds is fine for a profile; we don't cache the balance."
4. Stampede: "a single-flight lock and TTL jitter for popular keys."
5. Hot keys: "a local cache for a couple of seconds, and replicate the key if needed."

🎯 The phrase that marks a senior: "A cache is a trade-off between freshness and load. Let's decide how much staleness we're willing to tolerate." After that, the interviewer usually nods.

Common mistakes

• Proposing a cache and saying nothing about invalidation.
• Updating the key on write instead of deleting it (races).
• Setting keys without a TTL ("we'll delete them by hand").
• Giving the same TTL to keys warmed up in bulk.
• Caching things where staleness is unacceptable (money, stock at purchase time).
• Thinking sharding solves the hot key problem.

If you can name the race, the stampede, and the hot key, and give a defense with its price for each, you've got caching covered in the interview.