Rate limits

Rate limits apply per account and are shared by all its keys. They depend on your plan:

Plan	Requests per minute	Tokens per minute
Starter	300	300,000
Team	1,000	1,000,000
Scale	1,000	1,000,000
Enterprise	Custom	Custom

The limits in force for your account are on the Limits page of the dashboard. Higher limits can be arranged with our team.

When you hit a limit

The request is refused with 429 and a rate_limit_error. It is not billed. When the response carries a retry-after header, wait at least that long; otherwise back off exponentially and retry.

Headers

Non-streaming responses carry x-ratelimit-user-* headers describing your account’s allowance, such as x-ratelimit-user-limit-requests and x-ratelimit-user-remaining-tokens. Read them to slow down before you reach 429. No other rate-limit headers are forwarded.

Budget exhausted, not rate limited

A key whose budget is used up also gets 429 with a rate_limit_error, but retrying will not help: the key stays refused until its 30-day budget window resets or the budget is raised. The message tells the two apart. In your retry logic, stop retrying when the message mentions the budget.

Backing off with the OpenAI SDKs

The SDKs already retry 429, 408 and 5xx responses twice, with exponential backoff, and honor retry-after. For batch jobs, raise the retry count rather than writing your own loop:

Python

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://<your-gateway>/v1",
    api_key=os.environ["OVR_API_KEY"],
    max_retries=5,
)

JavaScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://<your-gateway>/v1",
  apiKey: process.env.OVR_API_KEY,
  maxRetries: 5,
});

If you send many requests in parallel, keep the concurrency below your requests-per-minute limit divided by the typical request duration in minutes. Spreading traffic beats retrying it.