Without rate limiting, any client opens the API to thousands of requests per second and takes the server down with it. With a poorly calibrated limit, you cut a paying customer in the middle of checkout. A right-sized rate limit protects the backend and warns the client before refusing.

The algorithms and what each one measures

Fixed Window

Counts requests inside a fixed window, 100 per minute for example. The boundary creates the edge burst problem: 100 requests at second 59 of the window plus 100 at second 0 of the next one pass in two seconds.

Sliding Window Log

Keeps the timestamp of every request and counts only the ones inside the current window. Accurate at any point of the boundary, and memory consumption grows with request volume.

Token Bucket

The bucket fills at a fixed rate and every request spends one token. Tokens on hand allow the burst, and the refill controls the average ceiling per period. RFC 2697 (Heinanen and Baker, 1999) describes the three-color marker built on a token bucket, and API gateways copied that design.

// Token bucket with Redis
async function isRateLimited(
  key: string,
  maxTokens: number,
  refillRate: number
): Promise<boolean> {
  const now = Date.now()
  const bucket = await redis.hgetall(`ratelimit:${key}`)

  if (!bucket.tokens) {
    // First request creates the bucket
    await redis.hset(`ratelimit:${key}`, {
      tokens: maxTokens - 1,
      lastRefill: now
    })
    await redis.expire(`ratelimit:${key}`, 60)
    return false
  }

  // Refill tokens based on elapsed time
  const elapsed = now - Number(bucket.lastRefill)
  const refill = Math.floor(elapsed / 1000 * refillRate)
  const tokens = Math.min(maxTokens, Number(bucket.tokens) + refill)

  if (tokens <= 0) return true

  await redis.hset(`ratelimit:${key}`, {
    tokens: tokens - 1,
    lastRefill: now
  })

  return false
}

The Redis documentation ships a rate limiter recipe built on sorted sets, which covers the sliding window without scanning old logs.

The key decides who gets limited

  • By IP: stops bots and anonymous abuse. CGNAT and VPNs put legitimate users in the same bucket.
  • By authenticated user: the limit follows the plan the customer pays for.
  • By API key: one bucket per developer on a public API. One account's abuse does not take the others down.
  • Compound: a ceiling per IP on top of a ceiling per user. DDoS and account abuse handled in the same pass.

Headers the client reads before retrying

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 23
X-RateLimit-Reset: 1711234567
Retry-After: 30  // present on the 429

X-RateLimit-* became a market habit through the Twitter API documentation and still has no RFC behind it. The IETF is standardizing the field set in draft-ietf-httpapi-ratelimit-headers (Polli and Martinez, IETF HTTPAPI WG), which defines RateLimit and RateLimit-Policy with explicit window semantics. RFC 6585 (Nottingham and Fielding, 2012) created the 429 status and anticipates Retry-After in the response. Without that header, the client guesses the wait time and starts a retry loop against your own rate limit.

express-rate-limit with a Redis store

import rateLimit from 'express-rate-limit'
import RedisStore from 'rate-limit-redis'

const limiter = rateLimit({
  windowMs: 60 * 1000,
  max: 100,
  standardHeaders: true,
  legacyHeaders: false,
  store: new RedisStore({
    sendCommand: (...args) => redis.call(...args),
  }),
  keyGenerator: (req) => req.user?.id || req.ip,
  handler: (req, res) => {
    res.status(429).json({
      error: {
        code: 'RATE_LIMITED',
        message: 'Too many requests. Try again in a few seconds.',
        retryAfter: Math.ceil(req.rateLimit.resetTime / 1000)
      }
    })
  }
})

app.use('/api/', limiter)

The keyGenerator returns the user id when a session exists and falls back to the IP for anonymous traffic. A Redis store shares the counter across instances; an in-memory store makes each pod count alone, and the limit becomes max times the number of pods. The express-rate-limit documentation covers standardHeaders and the warnings about proxies behind a load balancer.

Per-route limits follow route cost

Login runs bcrypt and costs milliseconds per attempt; listing runs an index and costs microseconds. A single limit for both punishes the cheap flow.

// Login: 5 attempts per minute stops brute force
app.use('/api/auth/login', rateLimit({ windowMs: 60000, max: 5 }))

// Listings: 100 per minute
app.use('/api/products', rateLimit({ windowMs: 60000, max: 100 }))

// Upload: 10 per hour
app.use('/api/upload', rateLimit({ windowMs: 3600000, max: 10 }))

When the limit lands on your own customer

A 429 with Retry-After is the floor of the contract. On top of it, graceful degradation returns a cached version or partial data when the data exists, instead of a bare error. Chapter 21, Handling Overload, of the Google SRE Book (2016) points out that rejecting a request still consumes server resources and that degraded responses keep the service alive through the spike. The customer gets something usable, and the backend keeps CPU for the traffic that goes through.

References