Rate limit

What is a rate limit?

A rate limit is a cap on how many requests a client may send to an API inside a fixed time window. GitHub, for instance, allows 5,000 requests per hour for an authenticated app and 60 per hour for anonymous callers. Stripe allows up to 100 requests per second in live mode. Go past the ceiling and the server stops answering normally.

Providers set these limits to protect their own infrastructure and to share capacity fairly. Without one, a single buggy script or an aggressive backfill could starve everyone else, or knock the service over. The limit turns "as fast as you can" into "this fast, and no faster".

You usually meet a rate limit when an integration moves from test to production: a trial run of a hundred records is fine, but the first full data pipeline sync of five years of history gets rejected.

How providers enforce them

How the limiter counts requests decides how bursts are treated.

Fixed window. The server counts requests per clock interval, say per minute, and resets the counter when the next minute starts. It is simple, but it has a boundary problem. Say the limit is 50 per minute. A client sends 50 in the last second of 10:00 and 50 more in the first second of 10:01. Each minute stays within its own count, yet the server handled 100 requests in about two seconds, double the intended rate.

Sliding window. To flatten that spike, the server blends the current and previous windows. Fifteen seconds into a fresh minute under the same 50 per minute limit, with 42 requests in the previous minute and 18 so far, the estimated rate is 42 times 45/60 plus 18, which is 49.5. That keeps the client just under the limit across the boundary.

Token bucket. A bucket holds tokens, one per request, and refills at a steady rate. Each request spends a token, and when the bucket is empty further requests are refused. Because tokens build up to the bucket's capacity while you are idle, the model allows a short burst up to the bucket size, then settles to the refill rate. Amazon API Gateway throttles this way, with the rate as the steady requests per second and the burst as the bucket capacity.

These limits usually live at an API gateway in front of the service, not in the application code.

Handling a 429 response

Cross the line and the server replies with status code 429, named Too Many Requests in RFC 6585. A well-behaved client reads that as a signal to slow down, not an error to show the user.

The first thing to read is the Retry-After header. RFC 6585 shows a server returning Retry-After: 3600, which tells the client to wait an hour. Many APIs also report how much headroom is left: GitHub returns x-ratelimit-limit, x-ratelimit-remaining, and x-ratelimit-reset, and the IETF is standardising equivalent RateLimit and RateLimit-Policy fields.

When no wait time is given, back off exponentially: pause briefly after the first rejection, then longer after each one after that, with a little random jitter so a fleet of clients does not retry in lockstep. This is where a retry policy and exponential backoff come in, and why the underlying operation should be idempotent. Idempotence is what makes a retry safe: repeating the call must not double-charge a customer or duplicate a record.