How to Rate Limit Calls to a Third Party API
Six real failure responses collected over three days. Only one of them meant slow down, and telling them apart matters more than any backoff algorithm.
Copy this line to your agent to fan out safely across many calls.
set up https://monid.ai/SKILL.md and cap concurrency when you run a batch
Almost every guide on this answers with exponential backoff and a jitter formula. That is the last ten percent of the problem. Between 2026-09-01 and 2026-09-03 this pipeline collected six distinct non-200 responses from six providers while researching other articles, and exactly one of them meant slow down. Backoff applied to the other five is either useless or actively harmful. This guide is about telling them apart, running through Monid, the OpenRouter for agent tools.
What does rate limiting a third party API actually mean?
Two different jobs that get the same name, and conflating them is why limiters end up badly tuned.
Staying inside a published quota
The provider documents a limit: so many requests per second, per minute, per key. This is arithmetic. You know the number, you enforce it, and you never see a 429. Nothing subtle happens here.
Staying inside an undocumented tolerance
The provider publishes nothing, or publishes a number that is not the real constraint. What actually exists is a threshold you discover by crossing it, and it can change without notice. Most third-party data APIs are in this category, and this is where the engineering lives. Node Unblocker covers what happens when you cross one without noticing.
The third thing people mean
Bounding cost. On per-call billing there may be no technical limit worth worrying about, and the ceiling you actually need is financial. That is a real requirement and it is not rate limiting, because the right control is a budget check before dispatch rather than a delay between requests.
Why the distinction decides the design
A quota needs a scheduler. A tolerance needs a feedback loop that reacts to what comes back. A budget needs an accountant. Building the first when you needed the second produces a system that is perfectly paced and still gets blocked.
📖 See also Do AI Agents Need a Rotating Proxy?
Why is exponential backoff the wrong first move?
Because it assumes the failure is about pace, and most failures are not.
What we actually collected
Six real responses, three days, six providers, all while doing other work:
| Response | What it meant | Retry? |
|---|---|---|
503 with retry_after: 60, attempts: 3 | Bot protection; already retried for us | Yes, after 60s |
502 from Cloudflare, twice | The origin is down | Later, not sooner |
500 "Something went wrong" | Provider-side fault | Once, then alert |
422 "Parameter 'symbol' is required" | Our request is wrong | Never |
400 listing six missing fields | Our request is wrong | Never |
| HTML instead of JSON | A redirect; not an API response | Never |
One of the six means slow down. Two mean the provider is having a bad day. Three mean we sent something invalid and no amount of waiting will fix it.
What backoff does to each
On the 422 and 400 it retries a request that is deterministically wrong, burning the retry budget and delaying the error that would have told a developer to fix the parameter name. On the 502 it adds load to an origin that is already failing. Only on the 503 is it doing something useful, and there the server told us exactly how long to wait, so the formula is not needed.
That is the case against reaching for backoff first: in the one situation where it helps, the response usually contains the answer.
The response worth studying
{
"error": "Service temporarily unavailable - site protection blocking all proxies",
"block_type": "retryable_protection",
"attempts": 3,
"retry_after": 60,
"status": "blocked"
}
attempts: 3 says the provider already retried on our behalf, so an aggressive client-side retry would be the fourth, fifth and sixth attempt against a target that has said no three times. retry_after: 60 removes the guesswork. block_type distinguishes a temporary block from a permanent one. A limiter that reads these three fields beats any formula that ignores them.
The classification, in code
def disposition(resp):
if resp.status == 429:
return "wait", retry_after(resp) or 60
if resp.status == 503 and body(resp).get("block_type") == "retryable_protection":
return "wait", body(resp).get("retry_after", 60)
if resp.status in (500, 502, 504):
return "provider_fault", None # retry ONCE, then alert
if resp.status in (400, 422):
return "our_bug", None # never retry, page a human
if not resp.headers.get("content-type", "").startswith("application/json"):
return "not_an_api_response", None # a redirect or an error page
return "ok", None
The last branch is the one people leave out. Twice this week a call returned HTML because a redirect was followed, and a client that only checks status codes parses it as JSON and reports a mystery error, which is the same silent-failure shape as an endpoint returning 26 empty fields and as a crawl returning empty Markdown.
How do you build a limiter that respects the server?
Three steps, and the first is the one that matters most.
For agents
Grab an API key at app.monid.ai, then paste this to your agent and hand it the key:
set up https://monid.ai/SKILL.md
It learns the whole discover, inspect, run workflow itself. More in the agent quickstart.
For humans
npm install -g @monid-ai/cli
monid keys add -k <your-api-key> -l main
Step 1. Cap concurrency before you tune delays
What it does. Bounds how many requests are in flight at once, which is the control providers actually feel.
The call. No endpoint. A semaphore:
sem = asyncio.Semaphore(5)
async def fetch(url):
async with sem:
return await client.get(url)
Five in flight is a defensible default for a third-party API you do not own. Requests per second is the metric people tune and concurrency is the one that causes the damage, because a burst of two hundred simultaneous connections looks like an attack regardless of the average rate over a minute.
What it costs. Nothing, and it prevents most of what backoff is invented to recover from.
Step 2. React to what comes back
What it does. Turns the classification above into behaviour.
The call. No endpoint:
kind, wait = disposition(resp)
if kind == "wait":
await asyncio.sleep(wait) # the server's number, not yours
shrink_concurrency() # and slow the whole pool, not one call
elif kind == "our_bug":
quarantine(request); alert() # never retried
shrink_concurrency is the part usually missing. If one call is being limited, every other call in that pool is about to be too, so the reaction belongs at the pool level. Halve on a limit signal and recover slowly, which is additive-increase multiplicative-decrease, the same idea TCP has used for forty years.
Step 3. Bound spend, separately
What it does. Handles the constraint that per-call billing actually creates.
The call. No endpoint, and this is deliberately not a rate limit:
if spent_today + estimated_cost > DAILY_CAP:
stop() # do not slow down, stop
A runaway loop is not fixed by pacing; pacing just spends the money more slowly. The check belongs before dispatch and it should halt rather than delay. This is the piece that catches a crawl with no excludePatterns, which is how an unbounded site crawl turns into an unbounded bill.
Give this to your agent
Set up https://monid.ai/SKILL.md, and then use Monid to run these 200 lookups with at most 5 in flight, honour any retry-after you get back, and stop entirely if the run would cost more than a dollar.📖 See also What a Blocked Scraper Actually Returns
What should you do when there is no documented limit?
Most of the time there is not one, so this is the normal case rather than the edge case.
Start low and stay there
Begin at a concurrency you are confident is polite, and only raise it if you have a reason. The instinct is to find the ceiling by probing for it, which means deliberately triggering the exact failure you are trying to avoid, on a provider who will remember.
Read the pattern of failures, not individual ones
A single 503 is weather. A 503 rate that climbs with your concurrency is a limit. Log the disposition of every response and watch the rate, because the individual failures are uninformative and their distribution is not.
Treat a working call as evidence too
Latency rising under load is the earliest limit signal available, and it arrives before any error does. If median latency doubles when you go from five to twenty in flight, you have found the tolerance without crossing it.
Ask, when the volume justifies it
For sustained high volume the answer is usually a conversation with the provider rather than a cleverer client. Documented limits and a higher tier exist for exactly this, and discovering the tolerance empirically is what you do when the volume does not justify a relationship. The same crossover shows up in the real cost of scraping YouTube yourself.
Which endpoint should I use for which job?
There is no rate-limiting endpoint, and there could not be: the control has to live in your client, next to the work being dispatched. What a per-call layer changes is which problem you have.
| Situation | Without a routing layer | With per-call access |
|---|---|---|
| Many providers | A quota and a key per provider | One key, one place to cap concurrency |
| Adding a provider | New limit to learn and enforce | Same client, same limiter |
| The binding constraint | Requests per key per window | Spend per run |
| Failure shapes | Different per provider | Still different per provider |
That last row is the honest one. A routing layer normalises access; it does not normalise what a target does when it dislikes you. The six responses at the top of this article came through one interface and still carried six different shapes, which is why the classification function exists rather than being handled for you.
Endpoints worth pairing with a limiter are the batch-shaped ones, where one call replaces many: context.dev/web/search returns results with content in a single call, and hunterio/multi-domain-search surveys many companies at once. Fewer, larger calls are a better answer to rate limiting than faster, smaller ones, which is the same search-then-enrich shape argued in the leadership contacts guide.
When does none of this help?
Three cases.
You are blocked, not limited. A challenge page returned with HTTP 200 is not a pacing problem, and slowing down will not fix a request whose TLS fingerprint is the issue. That is the subject of the proxy guide, and the failure looks nothing like a 429.
The provider is down. Two 502s minutes apart is an outage. Retrying faster makes it worse and retrying at all is optional; alerting is the useful action.
Your request is invalid. Three of our six failures were our own malformed requests, and every one of them named the field. A retry loop around a 422 converts a five-second fix into a slow, expensive mystery.
And the disclosure: this is Monid's blog and we sell per-call access to the endpoints referenced here. We do not sell a rate limiter, the code in this article is code you write yourself, and the table above says plainly that a routing layer does not normalise failure shapes. What it genuinely changes is that the constraint becomes spend, which is easier to bound than a quota per provider because there is one number to watch.
Conclusion
Rate limiting a third-party API is mostly a classification problem wearing an algorithm's clothes. Of six real non-200 responses collected across three days and six providers, one meant slow down, two meant the provider was failing, and three meant we had sent an invalid request. Exponential backoff is right for one of those six, and in that case the response contained retry_after: 60, so the formula was not needed anyway.
So the order is: cap concurrency at something polite before tuning anything, classify every response and react at the pool level rather than the request level, never retry a 4xx that named a field, check the content type so a redirect does not get parsed as data, and bound spend with a hard stop rather than a delay.
Free next step: log the disposition of every response your pipeline gets for a week, then count them. Most teams discover their retry budget is being spent almost entirely on requests that were never going to succeed. Start at monid.ai.
FAQ
What is the difference between a 429, a 503 and a 403?
A 429 is the unambiguous one: too many requests, slow down, and usually accompanied by a Retry-After header you should honour rather than guess around. A 503 means unavailable and can mean either an overloaded server or bot protection, so it needs the body inspected before deciding; ours carried block_type and retry_after and was clearly the latter. A 403 is refusal rather than pacing, and retrying it at any speed is pointless because the server has decided about you rather than about your rate.
Should you use a token bucket, a leaky bucket, or a semaphore?
Start with a semaphore, because concurrency is what providers actually feel and it is three lines of code. A token bucket is the right upgrade when you have a documented requests-per-second budget and want to allow short bursts within it. A leaky bucket smooths bursts into a constant rate, which is what you want when the provider is fragile rather than strict. Most teams reach for a bucket algorithm first, tune it carefully, and are still firing two hundred concurrent connections because the two controls are independent.
How do you rate limit across multiple workers?
The limiter has to live outside the process, because five workers each capped at five in flight is twenty-five concurrent requests, not five. A shared counter in Redis with a short TTL is the usual answer and is enough for nearly everyone. The failure to avoid is autoscaling behind a per-process limiter, where your effective rate silently scales with your worker count and the provider experiences a load spike every time your queue grows.
How many times should you retry before giving up?
Think in budgets rather than counts: cap retries as a fraction of total requests, commonly around ten percent, so a broad outage cannot triple your traffic exactly when the provider is struggling. Per request, two or three attempts is plenty for transient faults, and beyond that you are not recovering, you are queueing. Give up quickly on anything classified as our own bug, and remember that a provider which already reported attempts: 3 has spent your budget before the response even reached you.
Last updated September 2026.
