Design a Rate Limiter

Design a rate limiter that restricts the number of requests a client can make to an API within a time window. Support multiple limiting strategies and work correctly in a distributed environment.

Capacity Estimation

For 10M users, each making up to 100 req/min:

• Peak QPS: 10M × 100 / 60 ≈ 16.7M requests/sec
• Counter storage: 10M users × 1 counter × 8 bytes ≈ 80 MB in Redis — trivially fits in memory

Architecture

API Request
    │
    ▼
[Rate Limiter Middleware]
    │  checks Redis
    ├── Allowed → forward to API handler
    └── Denied  → 429 Too Many Requests

[Redis Cluster]
  key: ratelimit:{userId}:{window}
  value: request count
  TTL: window size

Integration

The rate limiter is middleware, not an API itself. Response headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1735689600
Retry-After: 37    (only on 429)

Redis Data Model

Fixed window counter:

SET ratelimit:user123:1735689600 0 EX 60
INCR ratelimit:user123:1735689600

Sliding window log (more accurate, more memory):

ZADD ratelimit:user123 <timestamp> <requestId>
ZREMRANGEBYSCORE ratelimit:user123 0 <now - window>
ZCARD ratelimit:user123

Use a Redis Cluster to distribute the keyspace. Each rate limiter instance talks to the same Redis cluster, so limits are enforced globally across all API servers.

• Fixed window is simple but allows burst at window boundary (50 req at end of window + 50 at start of next = 100 in 2 seconds).
• Sliding window log is accurate but uses more memory (one entry per request vs. one counter).
• Token bucket smooths bursts and is the most flexible — used by most cloud providers.

Redis is the single source of truth. Use Redis Cluster for horizontal scaling and Redis Sentinel for failover.

If Redis is unavailable: fail open (allow all requests) or fail closed (deny all). Most APIs choose fail open to avoid an outage becoming a complete blackout.