Lesson

APIs and Rate Limiting

The intuition: a club with one entrance

Picture a nightclub. Guests don't wander into the kitchen or the bar through the service doors: everyone comes in through a single entrance, where a bouncer checks tickets and doesn't let the whole crowd in at once. If there are already 200 people inside, newcomers hear: "Wait 10 minutes." That beats letting everyone in and bringing the place down.

A system works the same way:

The API is the contract for how guests deal with the club: the menu, the rules, how to order.
The API Gateway is the entrance with the bouncer.
Rate limiting is the rule "no more than N people per minute".

Let's go through them one by one. All three come up in almost every interview.

REST vs gRPC vs WebSocket / long polling

The main question here: who starts the exchange, and how often.

Approach	How it works	When to use	Downsides
REST (HTTP + JSON)	Request → response	Public APIs, CRUD, browsers and mobile apps	Verbose JSON, no server push
gRPC (HTTP/2 + Protobuf)	Binary calls, streaming	Service-to-service traffic inside the company, high RPS, strict contracts	Works in the browser only via a proxy, responses aren't human-readable
WebSocket	A persistent two-way connection	Chats, online games, stock tickers, collaborative editors	Stateful connections are harder to balance and scale
Long polling	The client asks, the server holds the request until there's data	Rare events, a simple fallback when WebSocket isn't available	Extra reconnects, higher latency

💡 In an interview it sounds like this: "Externally, REST — every client understands it. Between services, gRPC — it's faster and gives a typed contract. WebSocket for the chat, with long polling kept as a fallback."

Pagination: offset vs cursor

Offset — LIMIT 20 OFFSET 10000. It's simple, and it lets you jump straight to page 500. But the database has to read 10,020 rows and throw away 10,000 of them. There's a second problem too: if a new post is added to the top while the user is scrolling, every item shifts. The user then sees duplicates or misses entries.

1050

1041

1032

1023

offset=4 ↓ 1014

1005

996

987

Offset: the database reads the first 4 rows and throws them away. The deeper the page, the more expensive the query

Cursor — "give me 20 records after the last one I saw": WHERE id < 101 ORDER BY id DESC LIMIT 20. The database uses the index to jump straight to the right spot, and the cost doesn't depend on depth. Shifts in the feed don't hurt either.

1050

1041

1032

1023

cursor ↓ 1014

1005

996

987

Cursor: jump via the index right past the last id seen and read only what's needed

	Offset	Cursor
Speed on deep pages	Degrades linearly	Stable
Jump to page N	✅ Yes	❌ No
Stability under inserts	❌ Duplicates and gaps	✅ Stable
Where it fits	Admin panels, small tables	Feeds, infinite scroll

The cursor is usually handed to the client as an opaque string, for example base64 of {id, created_at}. That way you can change the format without breaking clients.

Rate limiting: three algorithms

A limit protects you from bots, from a buggy client firing requests in a loop, and from one greedy client eating everyone else's resources.

1. Fixed window. Keep a counter for each minute, say the key user:42:12:05, and INCR it. Once the counter exceeds the limit, respond with 429. It's simple and cheap, but it has a window boundary problem. With a limit of 100/min, you can send 100 requests at 12:05:59 and another 100 at 12:06:00 — that's 200 in one second.

2. Sliding window. The window "slides" along with time. Two variants:

log — store the timestamp of every request (a sorted set in Redis). Exact, but uses a lot of memory;
counter — weigh the current and previous windows: prev × (fraction of the previous window inside the sliding one) + current. Nearly exact and cheap.

3. Token bucket. Picture a bucket that holds 10 tokens, with 1 new token dropping in every second. Each request takes one token. Bucket empty — the request gets a 429.

Time	Tokens before	Requests	Result
0 s	10	10 (burst)	All pass, 0 left
1 s	1	3	1 passes, 2 → 429
5 s	4	2	Both pass, 2 left

A token bucket allows short bursts (the bucket's capacity) but holds the average rate (the refill rate). That's why AWS, Stripe and most gateways use it.

Algorithm	Accuracy	Memory	Bursts
Fixed window	Low at the boundaries	1 counter	Up to 2× where windows meet
Sliding window	High	Log — a lot, counter — little	Smoothed out
Token bucket	Good	2 numbers: tokens and time	Controlled, up to capacity

Where to store the counters

If every instance keeps its counters in its own memory, then with 5 instances a client effectively gets a 5× limit. So counters are stored centrally in Redis: it's fast and supports TTLs and atomic INCR. For a token bucket, checking and taking a token are packed into a Lua script so they run atomically, with no races.

The limit is checked at the entrance: the Gateway asks Redis and only then lets the request through to the services

Trade-offs worth saying out loud:

Redis adds ~1 ms to every request and becomes a dependency itself.
If Redis goes down, you need a policy: fail-open (let everyone through — the service stays up, but unprotected) or fail-closed (reject everyone — safe, but it hurts users). Fail-open is the more common choice.

API Gateway

It's a single entry point that takes on cross-cutting concerns so the services don't have to:

authentication (JWT validation);
rate limiting;
routing /users/* → User Service;
TLS termination, logs, metrics, tracing;
translating REST outside into gRPC inside.

Downsides: it's one more network hop, a potential single point of failure (so you run several instances behind a load balancer), and a temptation to put business logic there. Don't.

What a 429 response looks like

GET /api/v1/feed?cursor=eyJpZCI6MTAxfQ&limit=20 HTTP/1.1
Host: api.example.com
Authorization: Bearer eyJhbGciOi...

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 12
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1718000012

{"error": "rate_limited", "message": "Too many requests", "retry_after": 12}

Retry-After tells the client how many seconds to wait before retrying. A good client waits and uses exponential backoff with jitter instead of hammering the server again and again.

How it sounds in an interview

"The public API is REST with cursor pagination; internally, gRPC. At the entrance there's an API Gateway: it checks the token and applies a token bucket per user_id, 100 requests a minute with bursts up to 20. Counters live in Redis, with atomicity via Lua. If Redis is unavailable — fail-open, so we don't take the service down. When the limit is exceeded — a 429 with Retry-After."

Common mistakes

Storing counters locally on each instance: the limit "multiplies" by the number of servers.
Offset pagination for an infinite feed: deep pages get slow, and the user sees duplicates.
WebSocket where REST would do: complexity for the sake of rare updates.
Limiting by IP only: a whole office can sit behind one NAT. It's better to limit by user_id or API key and keep IP for anonymous requests.
Not stating what happens when Redis fails. The interviewer will almost certainly ask.

You've just built the whole "club entrance": how guests communicate, the bouncer and the rules of the line. It's a ready-made block for any design in an interview 💪