APIs
Rate limiting: code and snippet reading
Crux Read real limiter snippets — a token-bucket refill, a Redis INCR race, a sliding-window counter, a sorted-set log — and pick the behaviour or the highest-leverage fix.
Your altitude — climbing toward senior
ZeroJuniorMiddleSenior
You are at senior altitude — in orbitA limiter is correct or it is theatre, and the difference lives in a few lines of refill math and Redis commands. Read each snippet, predict its behaviour under concurrency, and pick the fix a senior would make first.
Goal
Practise the loop you run when reviewing a limiter: trace the refill, spot the race, check the boundary math, and reach for the atomic, shared-counter fix.
Snippet 1 — the token-bucket refill
# Per-request, in the app process. tokens/ts are read from Redis with GET, written back with SET.
def allow(key, rate, capacity, cost=1):
tokens, ts = read(key) # last stored tokens + timestamp
now = time.time()
tokens = min(capacity, tokens + (now - ts) * rate) # refill
if tokens >= cost:
tokens -= cost
write(key, tokens, now) # GET ... then SET (two round trips)
return True
write(key, tokens, now)
return False Quiz
Completed
The refill math is correct in isolation. What breaks when two requests for the same key arrive concurrently across two app nodes?
Heads-up min(capacity, ...) caps the refill, so a long idle period just tops the bucket up to capacity. The real defect is the non-atomic read-modify-write racing across nodes.
Heads-up Small clock skew slightly perturbs refill but is not the core bug. The core bug is that two concurrent GET-then-SET cycles interleave and double-spend, regardless of clock.
Heads-up Redis serializes each individual command, but GET and SET here are two separate round trips with app logic in between. Another request can read the stale value before this one writes — that gap is the race.
Snippet 2 — the Redis fixed-window counter
def allow(key, limit, window_s):
n = redis.incr(key) # atomic increment
if n == 1:
redis.expire(key, window_s) # set TTL only on first hit
return n <= limit Quiz
Completed
INCR is atomic, so what is the latent failure in this fixed-window limiter?
Heads-up INCR is fully atomic — that part is fine. The bug is the gap between INCR and the separate EXPIRE, which can leave the key without a TTL forever.
Heads-up n <= limit correctly admits exactly limit requests (the first INCR returns 1). The off-by-one is not the defect; the missing-TTL race is.
Heads-up It does leak the 2x boundary burst, but that is inherent to fixed window, not a bug in this code. The code-level defect is the INCR/EXPIRE atomicity gap that can strand a TTL-less key.
Snippet 3 — the sliding-window counter
# limit = 100 / 60s. Two fixed-window counters: this minute and the previous one.
def estimate(prev_count, curr_count, elapsed_in_window_s, window_s=60):
overlap = (window_s - elapsed_in_window_s) / window_s # fraction of prev window still in view
return curr_count + prev_count * overlap
# Example: 18s into the current minute, prev=80, curr=12 Quiz
Completed
For the example values (18s in, prev=80, curr=12, limit=100), what does estimate return and is the request admitted?
Heads-up Adding prev in full ignores the overlap weighting that is the whole point. The previous window contributes only the fraction still inside the rolling view: 80*0.70 = 56, not 80.
Heads-up Dropping prev entirely is just a fixed window again and reintroduces the boundary burst. The counter's accuracy comes from weighting the previous window by its remaining overlap.
Heads-up The overlap is the fraction of the previous window still in the trailing view, which is (60-18)/60 = 0.70, not 18/60. As the current minute advances, the previous window should weigh less, not more.
Snippet 4 — the sliding-window log in Redis
def allow(key, limit, window_s):
now = time.time()
pipe = redis.pipeline()
pipe.zremrangebyscore(key, 0, now - window_s) # drop entries older than the window
pipe.zadd(key, {str(uuid4()): now}) # record this request
pipe.zcard(key) # count what remains
pipe.expire(key, window_s)
_, _, count, _ = pipe.execute()
return count <= limit Quiz
Completed
This log-based limiter is exact and atomic (MULTI/EXEC pipeline). What is the real cost you sign up for, and the subtle behaviour of this ordering?
Heads-up ZREMRANGEBYSCORE deterministically drops everything older than now - window, so the count is exact over the rolling window. The cost is memory per request, not accuracy.
Heads-up Members must be unique or two requests at the same timestamp would collide into one entry. A UUID (or timestamp plus a counter) is correct precisely to avoid that collision.
Heads-up A Redis MULTI/EXEC pipeline runs the commands atomically with no interleaving, so the set is consistent. The price is the per-request memory of the log, which is why the counter is preferred at scale.
Recap
Limiter bugs live in a few lines: app-side read-modify-write double-spends across nodes, so the check must run atomically inside Redis; INCR-then-EXPIRE can strand a TTL-less key forever; the sliding-window counter weights the previous window by its remaining overlap, not in full; and the sliding-window log is exact and atomic but pays one sorted-set entry per request. Trace the concurrency, check the boundary math, and push the whole decision into one atomic, shared step.