Feature

Rate Limits

Cap requests per minute and tokens per minute on any key, with limits you set instead of ones a provider imposes on you. Over the limit, a request returns 429 with a Retry-After header before it costs anything.

How it works

Set a ceiling on the key

Configure requests-per-minute, tokens-per-minute, or both, at the key level or overridden per model.

Every request is checked first

Aniron evaluates the limit before routing to a model, so a rejected request never reaches a provider and never incurs cost.

Standard 429, standard retry

Over the limit, the response is a 429 with Retry-After, matching the OpenAI API shape your client's existing retry logic already handles.

create-key.ts

const key = await aniron.keys.create({
  name: 'public-api-proxy',
  rate_limit: {
    requests_per_minute: 300,
    tokens_per_minute: 150_000,
  },
});

// A request over either limit returns 429 with
// a Retry-After header, before it reaches a model.

429 Standard response code

Same status and Retry-After header shape as the OpenAI API, so existing client retry logic works without changes.

When to use rate limits

Protect a downstream system

Cap tokens-per-minute on a key feeding a service that cannot handle bursts of concurrent model output.

Contain a public-facing proxy

Put a request-per-minute ceiling on a key exposed behind a public API so one client cannot exhaust the account's throughput.

Slow spend without hard-stopping it

Pair a rate limit with a per-key budget so a key throttles smoothly under load instead of hitting its spend cap all at once.

Load-test safely

Set a conservative limit on a load-testing key so a misconfigured test script cannot generate an unbounded number of billed requests.

Frequently asked questions

Are rate limits set per key or per account?

Per key by default, so one integration hitting its ceiling does not throttle another. You can also set an account-wide limit as a backstop across all keys.

What response do I get when a limit is hit?

A standard 429 status with a Retry-After header telling you how many seconds to wait, the same shape the OpenAI API uses, so existing retry logic works unmodified.

Can I limit requests and tokens separately?

Yes. Requests-per-minute caps request volume regardless of size, and tokens-per-minute caps throughput regardless of how many requests that took. You can set either, both, or neither.

Can different models on the same key have different limits?

Yes. You can set a default limit for a key and override it for specific models, useful when one model in your chain is more expensive or more prone to upstream throttling.

Do rate limits interact with model fallbacks?

Yes, a 429 from Aniron itself is not a fallback trigger since it means your own configured ceiling was hit, not a provider failure. A 429 from the upstream provider is what triggers a fallback.

Related resources