Rate Limits

How limits are scoped

Limits apply per API key, not per account — two keys on the same account each get their own request and token budget. This is why splitting traffic across per-service or per-environment keys (as recommended in Authentication) also isolates one noisy service from throttling another.

Response headers

Every response — successful or not — includes headers describing the current window:

HTTP/1.1 200 OK
x-ratelimit-limit-requests: 500
x-ratelimit-remaining-requests: 497
x-ratelimit-limit-tokens: 200000
x-ratelimit-remaining-tokens: 184213
x-ratelimit-reset-requests: 12s
x-ratelimit-reset-tokens: 41s

Track x-ratelimit-remaining-tokens client-side to throttle proactively — it's cheaper to slow down before a 429 than to recover from one.

Exceeding the limit

Once either the request or token limit is hit, further requests in that window return 429:

{
  "error": {
    "message": "Rate limit reached for requests. Limit: 500/min. Retry after 12 seconds.",
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded"
  }
}

Backoff guidance

Use exponential backoff with jitter, honoring Retry-After when present:

async function withBackoff(fn, maxAttempts = 5) {
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (err.status !== 429 || attempt === maxAttempts - 1) throw err;
      const retryAfter = Number(err.headers?.['retry-after']) || 2 ** attempt;
      const jitter = Math.random() * 0.3 * retryAfter;
      await new Promise((r) => setTimeout(r, (retryAfter + jitter) * 1000));
    }
  }
}

A better default: fallbacks

Client-side backoff adds latency — the request waits, then retries the same rate-limited model. Setting a fallbacks array instead routes immediately to a different model with its own independent rate limit, so throughput isn't gated by a single model's window. See Model Fallbacks.