Rate limits, quotas and usage charges

A rate limit caps how often you can ask. A spend limit caps how much you can use. Both are set by the service, and some you can set yourself. Knowing which one you hit tells you what to do next.

Rate limits: how fast

¹ OpenAI's documentation calls rate limits "restrictions" on how often a user or client may use the service within a set time. A shop door with a queue is a fair picture. The door still opens, but only so many people go through at once.

Why have them?¹ The same page says "They help protect against abuse or misuse of the API." It also gives fair access for everyone as a reason.

What gets counted

You might expect only requests are counted. Often more is.¹ OpenAI says "Rate limits use metrics such as RPM (requests per minute), RPD (requests per day), TPM (tokens per minute)" and others. A token is a small chunk of text that AI models read and write.

So a request can be refused because it was too long, not because there were too many.

¹ The page also says "Rate limits vary by the model being used." One model can have a different allowance from another, so check the maker's own limits page for yours.

Who shares the limit

Limits often belong to the account, not to one person.¹ OpenAI says they're set per organisation and per project, "not user level."² Anthropic says they're set at the organisation level.

That means a colleague's busy script can use up the allowance your own tool needs.

Bursts

Even a steady average can trip a limit.² Anthropic notes that "Short bursts of requests can exceed the limit and trigger rate limit errors." Spread requests out rather than sending a hundred in one second.

What the refusal looks like

¹ OpenAI says "When a request exceeds a temporary rate limit, the API returns a 429 error." For this reason, 429 is the number to watch for. Our guide to reading error codes covers it.

Waiting a little and trying again often fixes a temporary limit. But it won't fix every 429.¹ The same documentation warns: "Don’t retry quota, billing, or other errors that require you to take action."

Usage charges: how much

Whether you're charged, and how, is up to each maker. Check how yours bills before you build on it. Each maker publishes its own prices on its own page. Those change, so read the live page and note the date and currency.

How a bill gets bigger than expected

The common causes are plain.

• A script that loops and keeps sending requests.
• A leaked key that someone else is using. See how to protect an API key.
• Longer prompts than you thought, if your maker's pricing page counts size.
• Several people or tools sharing one account.

Setting your own ceiling

You can often set a spend limit yourself.³ OpenAI advises: "Set monthly spend limits for your organization or projects". It also says to "enable hard enforcement". The aim is to help limit unexpected costs from a leaked key. It adds a warning: "Spend alerts alone don't stop API traffic." An alert only tells you. A limit with hard enforcement can stop traffic.

There is a catch.³ OpenAI says "Hard enforcement can block legitimate requests". It also says "spend may slightly exceed the limit because enforcement isn't instantaneous."

² Anthropic says "You can also set your own spend limit below your tier’s cap to control costs".² Its tier cap is "which is the maximum your organization can spend on the API each calendar month."

Once a limit is reached, requests stop until it's raised or resets. That's the point of it, but it also means a customer-facing tool could go quiet. Pick a limit with that in mind, and tell the people who rely on the tool.

Keep customer data, passwords, keys and card numbers out of prompts unless your business's own policy allows it.