Rate limits

When to use this pageCopy Anchor Link

You got a 429, 402, 529 or 503, or you are building a client that should retry well. Every error type is on Errors.

The limitsCopy Anchor Link

On the OpenAI-compatible API, the same limits return the OpenAI format: 429 rate_limit_exceeded, 402 insufficient_quota, and 503 overloaded.

Requests per minuteCopy Anchor Link

• Each key can make 1,000 requests per minute unless EZAT Router has set another limit for it. Keys don't share this limit.
• POST /v1/messages, POST /v1/messages/count_tokens and POST /openai/v1/chat/completions count. Model lists don't.
• The count resets at the start of each minute. Refused requests count too, so retrying in a tight loop keeps the key over the limit until the minute ends.
• Over the limit, the gateway answers 429 with retry-after: 30 and the message "This API key exceeded its requests-per-minute limit". The request isn't charged.

The dashboard can't change a key's limit. To ask for a different one, email support@ezat.io with the key's name and the start of the key as the dashboard shows it (ezat_sk_…).

Spend limitsCopy Anchor Link

A spend limit caps what one key can spend in a period. Set it when you create the key: API keys → Create key → Spend limit, with an Amount (USD) and a Period.

• The gateway checks the limit before each request. Once the key's spending in the period reaches the limit, requests get 402 with a message such as "This API key reached its monthly spend limit of …".
• A request that starts under the limit finishes, so spending can end slightly above it.
• The dashboard can't change a key's limit after it is created. Create a new key with the limit you want, move your app to it, then revoke the old key.

See API keys for the rest of a key's settings.

Organization balanceCopy Anchor Link

All keys in an organization spend one prepaid balance. When it is zero or below, every key gets 402 until someone adds credit. See How billing works.

CapacityCopy Anchor Link

Each model is served through one or more provider deployments. The gateway picks one for each request and, if it fails before the response starts, tries up to two others.

• When none can take the request, the gateway answers 529 overloaded_error with retry-after: 5 (503 overloaded on the OpenAI-compatible API).
• When the providers are rate limiting, it answers 429 "Rate limited upstream. Please retry shortly." with retry-after: 10.
• Neither is charged. Retry after retry-after.

A model that has no capacity at all shows "available": false in GET https://api.router.ezat.io/api/models. See Models.

HeadersCopy Anchor Link

There are no headers with your remaining quota (no x-ratelimit-* or anthropic-ratelimit-*).

RetryingCopy Anchor Link

The official Anthropic and OpenAI SDKs (Python and Node.js) already retry connection errors, 408, 409, 429 and every 5xx (529 and 503 included), twice by default, with exponential backoff, and they wait the retry-after time when the header is there. Change it with max_retries (Python) or maxRetries (Node.js), on the client or per request.

If you write your own client:

1. Retry only 429, 500, 529 and 503, and connection errors. Don't retry 400, 401, 402, 403, 404 or 413.
2. Wait retry-after seconds when the header is there.
3. Otherwise wait exponentially longer each time, with random jitter: for example 0.5 s, 1 s, 2 s, 4 s, up to about 8 s, each multiplied by a random factor between 0.75 and 1.
4. Stop after a few attempts and report the error with its request-id and x-request-id.
5. Send the retry exactly as before, with the same x-ezat-session header (or prompt_cache_key on the OpenAI-compatible API) if you use one. The gateway keeps a conversation on the provider that holds its prompt cache, so an identical retry keeps the cache and costs less.

Don't retry a 504 timeout_error as it is: it means a non-streaming request ran for 590 seconds, and the SDKs would send it again in full. Send long requests with "stream": true instead.

import random
import time


def with_retries(send, attempts=5):
    """send() returns (status, headers, body). Retries 429, 500, 503 and 529."""
    for attempt in range(attempts):
        status, headers, body = send()
        if status not in (429, 500, 503, 529) or attempt == attempts - 1:
            return status, headers, body
        retry_after = headers.get("retry-after")
        if retry_after:
            delay = float(retry_after)
        else:
            delay = min(0.5 * 2**attempt, 8) * random.uniform(0.75, 1.0)
        time.sleep(delay)

For AI agentsCopy Anchor Link

1. Per-key limit: 1,000 requests per minute by default; 429 with retry-after: 30 when over it.
2. On 429, 529 or 503, wait retry-after seconds, then retry with exponential backoff and jitter. The official SDKs do this already (2 retries by default; raise max_retries / maxRetries for batch jobs).
3. On 402, stop: the balance is used up or the key reached its spend limit. Tell the user to add credit at https://router.ezat.io/billing, or to create a key with a higher limit.
4. There are no remaining-quota headers. Pace requests yourself if you need to stay under the per-minute limit.
5. Keep the same x-ezat-session value (or prompt_cache_key) when retrying, so prompt caching keeps working.
6. Use "stream": true for long requests: non-streaming requests end with a 504 after 590 seconds.