Caching: Where and How

Intuition: the fridge in your kitchen

Imagine going to the store every time you want milk. That takes forever. So you keep milk in the fridge, and you only go to the store when you run out. A cache is that fridge: fast storage right next to whoever reads the data. The slow source, the database, is only needed on a miss.

The two problems are fridge problems too: the milk can go sour (the data is stale), and the fridge is small (you have to decide what to throw out). This whole lesson is about those two things.

Cache layers: from the user to the database

There can be a cache at every step of a request's path. The closer it is to the user, the faster the response and the harder that cache is to flush.

Layer	What it stores	Latency	How to invalidate
Browser	static assets, responses with Cache-Control	~0 ms	barely at all: wait out the TTL or change the URL (app.v42.js)
CDN	images, JS, sometimes public API responses	10–30 ms	purge through the provider's API
In-process memory cache	small hot data, configs	nanoseconds	every instance has its own copy, and they drift apart
Application cache (Redis/Memcached)	profiles, feeds, sessions	~1 ms	DEL key, shared by all instances
Database cache	buffer pool, pages in RAM	inside the DB	the DB manages it itself

In an interview, walk through the layers top to bottom in a single sentence: "I serve static assets through a CDN with a long TTL and versioned URLs, keep hot objects in Redis, and for really hot configs a local in-memory cache with a short TTL works." That shows maturity right away.

Cache-aside: the default strategy

The most common pattern. The application manages the cache itself:

1. A request comes in, and we look up the key in Redis.
2. Hit: return the response right away.
3. Miss: read from PostgreSQL, put the result in Redis with a TTL, return the response.
4. On a write, update the database and delete the key from the cache. The next read will pull in fresh data.

Why do we delete the key on a write instead of updating it? Two concurrent updates can write their values into the cache in the opposite order. Deleting is safer.

Let's do the math on why this matters

Say you have 10,000 reads per second. Redis responds in 1 ms, the DB in 10 ms, and the hit rate is 90%.

Metric	Calculation	Result
Requests to the DB	10,000 × 0.1	1,000 RPS instead of 10,000
Average latency	0.9 × 1 + 0.1 × (1 + 10)	≈ 2 ms instead of 10
At a 99% hit rate	10,000 × 0.01	100 RPS to the DB

Every extra "nine" in the hit rate takes another order of magnitude of load off the database. Say this out loud in an interview; it's a strong argument.

Four strategies compared

Strategy	How it works	Pros	Cons
Cache-aside	the application reads the cache, goes to the DB on a miss, and puts the result in the cache	simple; a cache outage doesn't take the system down; only needed data ends up in the cache	the first request is slow; cache logic is spread across the code; stale reads are possible
Read-through	on a miss, the cache goes to the DB itself (via a library or provider)	cleaner application code	the cache has to know how to read the DB; the first miss is still slow
Write-through	writes go to the cache, and the cache synchronously writes to the DB	the cache is always fresh	slower writes (two operations); data nobody reads ends up in the cache
Write-back (write-behind)	writes go only to the cache and are flushed to the DB asynchronously in batches	very fast writes; less load on the DB	risk of losing data if the cache dies before flushing; harder consistency

The rule of thumb: go with cache-aside by default. Use write-through when reads right after a write must be fresh. Write-back fits counters, likes, and metrics, where losing a couple of seconds of data isn't critical.

TTL: the expiration date

TTL (time to live) sets how long a key lives. It's your safety net: even if you forgot to invalidate a key, it disappears on its own once the TTL runs out.

• Short TTL (seconds): fresher data, but more misses.
• Long TTL (hours): higher hit rate, but users see old data for longer.
• Add jitter: TTL = 300 ± random(30). Otherwise thousands of keys expire at the same moment, and the whole load hits the DB at once.

Eviction: what to throw out of a full fridge

Memory is finite. When it runs out, the eviction policy kicks in. The most popular one is LRU (Least Recently Used): throw out whatever hasn't been accessed for the longest time.

On every read, the key moves to the head of the list. There's also LFU (throw out what's used least often). It does better when popularity is stable. In Redis, this is the maxmemory-policy allkeys-lru setting.

Warming up: the cold start

After a deploy or a Redis restart, the cache is empty, the hit rate is 0%, and all 10,000 RPS hammer the database. Warming up means filling the cache with hot keys ahead of time: top products, popular profiles, yesterday's data. Ramping traffic up gradually also helps, as does stampede protection: when a hot key expires, only one request goes to the DB for it (a lock or single-flight), and the rest wait.

How it sounds in an interview

"The workload is read-heavy, 100:1, so I'll put Redis in front using cache-aside. A 5-minute TTL with jitter, and a DEL of the key on writes. I expect a hit rate around 95%, which takes 20x the load off the DB. If Redis goes down, the system degrades but keeps working through the DB. For static assets, a CDN with versioned URLs."

Common mistakes

• Caching without a TTL. Stale data will live forever.
• Not naming the price. A cache always means eventual consistency, and you need to say so.
• Forgetting the cold start and the stampede. The interviewer will almost certainly ask, "What happens if the cache goes down?"
• Write-back for money. Payments can't be lost; only a durable write to the DB will do here.
• A local cache with no caveats. Explain that copies on different instances drift apart, so the TTL has to be short.

You can now answer "where," "how," "how long it lives," and "what to throw out." That's the whole skeleton of an answer about caching.