Browser Automation When the Site Has No API
Check the premise first. The site owner has no API, but somebody may already have an endpoint for it, and a browser is the most expensive answer available.
Copy this line to your agent to check whether an endpoint exists before it builds a browser.
set up https://monid.ai/SKILL.md and use monid discover to check whether an endpoint already covers this site
Every comparison of browser automation tools starts by accepting the premise, and the premise is often wrong. "This site has no API" almost always means "the site owner does not publish one", which is a different statement. Somebody else may already sell an endpoint for it, and a browser is the most expensive answer in the room. This guide is about testing the premise first, then choosing properly if it holds, running through Monid, the OpenRouter for agent tools.
What do you use when a site has no API?
Four options, and they are not a ranked list. They differ by an order of magnitude in cost and in how likely they are to still work next quarter.
Somebody else's endpoint
The one the comparison articles skip entirely. For popular targets, a third party has usually already built and maintained the extraction, and sells it per call. This is not the site's API and it does not need to be. It is somebody who solved the problem and is renting the solution.
It costs a fraction of a cent per call, requires no infrastructure, and survives the target's redesigns because maintaining that is the provider's job rather than yours. The catch is coverage: it exists for popular sites and not for obscure ones.
A generic extraction endpoint
One level down in specificity. Point a scrape endpoint at any URL and get clean Markdown or typed JSON back. Rendering, anti-bot handling and the exit address are included; the parsing is generic rather than tuned to that site.
This is the right answer for a site nobody has purpose-built for, which is most sites.
A hosted browser session
A real Chrome running on somebody else's infrastructure, driven by your Playwright or Puppeteer code over a websocket. You get full interaction: clicks, typing, file uploads, multi-step flows. You pay by the minute or the gigabyte, which is a different and much steeper cost curve.
A browser you run
Playwright or Puppeteer on your own machines, with your own proxies, your own restart policy and your own on-call. Maximum control, and the whole operating cost lands on your team.
The four options, side by side
| Aspect | Somebody's endpoint | Generic extraction | Hosted browser | Your own browser |
|---|---|---|---|---|
| Cost shape | Per call or per result | Per call | Per minute or per gigabyte | Servers plus salaries |
| Handles login flows | No | No | Yes | Yes |
| Survives a redesign | They fix it | Usually | Your code breaks | Your code breaks |
| Exit address | Included | Included | Included | You buy it |
| Time to first result | Minutes | Minutes | An afternoon | A sprint |
| Coverage | Popular sites only | Almost anything | Anything | Anything |
Read down the first row. The gap between the left two columns and the right two is roughly two orders of magnitude, and most people arrive at this question having already decided on the right-hand side. The same four-way sort applied to scraping rather than automation is in Web Scraping Tools: Which Kind Do You Actually Need?.
📖 See also Agentic Browser: When Your Agent Actually Needs One
Why is a browser the most expensive answer?
Three costs, and only one of them appears in any comparison table.
You are renting a computer, not a request
A scrape endpoint bills for a page. A browser bills for time or for bandwidth, because that is what it consumes: a Chrome process, memory, and every byte of every image, font and tracking script the page loads. A page that costs a fraction of a cent to fetch as Markdown can cost meaningfully more to render, because rendering it means downloading all of it.
That ratio is the single most useful thing to know before choosing, and it is why "just use a browser" is expensive advice at any volume.
The code is the fragile part, not the infrastructure
A browser script encodes the page's structure: this selector, then that button, then wait for this element. Every one of those is a promise the site never made. A redesign breaks the script, and the break is usually silent, because a selector that matches nothing produces an empty result rather than an error.
An endpoint moves that fragility to the provider. When the target redesigns, somebody else's parser breaks and somebody else fixes it, and you find out because the data kept arriving. The silent half of that failure, a request that succeeds and returns nothing, is the one worth alerting on, and we hit a live example of it in the Crawl4AI guide.
Detection is an arms race you did not sign up for
Automated browsers are detectable, which is why the stealth-plugin ecosystem exists. Adopting a browser means adopting that arms race: a patch here, a fingerprint tweak there, a plugin that worked last quarter. It is genuinely interesting engineering and it is almost never the thing your product is about.
How do you check whether an endpoint already exists?
Two commands, both free, and this is the step that makes the premise testable rather than assumed.
For agents
Grab an API key at app.monid.ai, then paste this to your agent and hand it the key:
set up https://monid.ai/SKILL.md
It learns the whole discover, inspect, run workflow itself. More in the agent quickstart.
For humans
npm install -g @monid-ai/cli
monid keys add -k <your-api-key> -l main
Step 1. Ask the catalog before you build
What it does. Searches over a thousand endpoints by description, so "does anything already read this site" becomes a question with an answer rather than an assumption.
The call.
monid discover -q "job listings from a job board"
monid discover -q "amazon product details by asin"
What comes back. A ranked list with provider, endpoint, price, billing shape and a verified flag. Running exactly this on 2026-08-30 turned up dedicated endpoints for Built In, Wellfound, Idealist and web3.career, plus Amazon search, product-detail and review endpoints. None of those sites publishes the API you would have gone looking for.
The result is equally useful when it is empty. Searching for eBay and Etsy coverage on the same day returned nothing specific to either, which tells you immediately that those two are generic-extraction jobs rather than purpose-built ones. That is a five-second answer to a question people spend an afternoon on.
What it costs. Nothing. Discovery never bills.
Step 2. Read the schema before you spend
What it does. Shows the exact fields and the current price, so you can judge coverage without a single billed call.
The call.
monid inspect -p builtin -e /search_jobs
What comes back. The input schema, the billing shape and the pricing note. If the fields are not the ones you need, you have learned that for free, which is the entire point of the step.
What it costs. Nothing.
Step 3. Run one, small
What it does. Replaces an estimate with a measurement.
The call.
monid run -p builtin -e /search_jobs -w --query '{"query": "data engineer"}'
What comes back. Running that on 2026-08-30 returned job records carrying id, title, url, company, company_url, company_logo, work_type, location, salary, experience_level and posted.
Note work_type and experience_level. Those are the board's own normalised fields, and reconstructing them from a rendered page means writing a classifier. That is the difference between an endpoint somebody maintains and a browser script you maintain, made concrete.
What it costs. A cent or so per call. Current figures at monid.ai/tools. There is a worked version of this whole check, applied to a live web question, in How I Gave My Agent Eyes on the Live Web.
📖 See also Which MCP Server Gives an AI Agent Live Web Data?
Give this to your agent
Set up https://monid.ai/SKILL.md, and then use Monid to before writing any browser code for this site, check whether an endpoint already covers it, read the schema, and tell me what fields it returns.What is the best antidetect browser?
Two products share the name and they are bought by different people for different reasons, which is why the ranked lists disagree with each other.
The multi-account browser
A desktop application that maintains many isolated browser profiles, each with its own fingerprint, cookies and proxy. The buyer is a person managing many accounts on one platform, and the product is isolation between them. This is the larger market and it is not an automation tool.
The automation-focused stealth stack
Libraries and patched browsers whose goal is that a programmatically driven Chrome looks like a human-driven one. The buyer is a developer whose script is being blocked. This is what people mean when they arrive from a scraping problem.
The honest position on the second one
It works, it is a moving target, and it is a poor foundation for anything that has to keep working without attention. The stealth ecosystem exists because detection improves; adopting it means committing to track that. If your product is the automation, that is a reasonable commitment. If your product is something else and the automation is plumbing, you are taking on maintenance in a domain you do not care about.
The alternative is to buy the exit address rather than disguise the browser, which is the argument we made with measured billing in Do You Still Need a Rotating Proxy in 2026?. Where a managed endpoint includes anti-bot handling and does not bill blocked requests, the detection problem stops being yours.
Which endpoint should I use for which job?
| Endpoint | What it does | Input | Output | Best for | Billing |
|---|---|---|---|---|---|
context.dev/web/scrape/markdown | Any page to Markdown | A URL | Markdown, metadata, JSON-LD | The site has no purpose-built endpoint | Per call, misses free |
context.dev/web/extract | Any page to typed JSON | URL plus a JSON Schema | Typed fields | You want fields, not prose | Per result |
context.dev/web/crawl | Walks a whole site | Start URL, depth, cap | Markdown per page | A section, not a page | Per page |
x402.browserbase.com/browser/session/create | A remote Chrome you drive | None | Session id and connect URL | The task needs clicking | Per call, then time |
| Site-specific endpoints | The site's data, parsed | Varies | Normalised fields | The site is popular | Per call or result |
Every row was verified with monid inspect on 2026-08-30. The table gives billing shape rather than figures; shape drives design and current numbers live on monid.ai/tools.
The ordering in that table is the decision procedure. Work down it and stop at the first row that covers your case, because each step down costs more than the one above it.
When do you genuinely need a browser?
Four cases, and they are real rather than a formality.
The task carries state. A logged-in session, a multi-step form, a cart, anything where step three depends on what happened in step two. No stateless fetch reproduces this and you should not try.
The action is the point. You are not reading a page, you are clicking something: submitting, uploading, confirming. Extraction endpoints read; they do not act.
Nobody has built for this site and the page is JavaScript all the way down. A generic extraction endpoint handles most rendered pages, but some applications only assemble their content after interaction, and there is no shortcut.
The site is defended and central to your business. Then the arms race is worth entering deliberately rather than by accident, and the trade-offs are set out in Your Scraper Is Blocked: What Actually Gets Through.
You are testing your own application. Playwright and Puppeteer are testing tools first. If that is the job, this whole article is about a different problem and the comparison articles that rank above us are the right reading.
There is also a case for the specialists. If browser automation is your product rather than your plumbing, a vendor whose entire business is browser infrastructure will out-run a generalist on session management, fingerprinting and concurrency, and you should buy from them.
And the disclosure: you are reading Monid's blog, we sell per-call access to tools, so the case we argue best is the one where the browser turns out to be unnecessary. When it is necessary, it is necessary, and we say so above.
Conclusion
The comparison you were about to run, which browser automation tool is best, is a comparison of the most expensive option against itself. Before running it, spend the free minute checking whether the site is already covered by somebody's endpoint, and the free minute after that reading the schema. On popular targets the answer is frequently yes, and the cost difference between that answer and a browser is roughly two orders of magnitude.
If the premise survives the check, the ordering still holds: generic extraction before a hosted browser, a hosted browser before one you run. Each step down that list buys capability you may not need and costs a computer rather than a request.
Free next step: run monid discover with a plain description of the site and the data you want. It costs nothing, it takes a minute, and an empty result is as useful as a full one because it tells you which kind of job you actually have. Start at monid.ai.
FAQ
What is nodriver and is it better than Selenium?
It is a successor to undetected-chromedriver that talks to Chrome over the DevTools protocol directly rather than through a webdriver, which removes a large and well-known detection surface. For a developer already committed to running their own browser it is a genuine improvement and worth using over the older stack. It does not change the economics discussed above: you are still operating a browser fleet and still buying your own exit addresses, and those are the costs that dominate.
Does puppeteer-extra-plugin-stealth still work?
It still helps and it has never been a solved problem, which is the honest answer. The plugin patches the specific signals that identify an automated Chrome, detection vendors adapt, and the plugin adapts back. Treat it as ongoing maintenance rather than a fix you apply once. If the site you are fighting is defended enough that stealth is required, the cheaper route is usually an endpoint whose provider is already fighting that battle across many customers.
Can a site tell you are running headless Chrome?
Yes, through a long list of signals: missing or inconsistent browser APIs, timing patterns, canvas and font fingerprints, and the way an automated session moves through a page. Modern headless mode closed many of the obvious gaps but not the behavioural ones. This is why the honest framing is not "can I hide" but "how long do I want to keep hiding", and the answer determines whether you should be running the browser at all.
How much does browser automation actually cost?
It bills on a different unit from everything else in this category, which is what makes it expensive rather than the headline rate. Extraction bills per page; browsers bill per minute or per gigabyte of traffic, and a rendered page pulls down every image, font and script it references. Before committing, price one representative task both ways: the same job as a scrape call and as a browser session. The ratio is usually larger than people expect, and it is the number that should decide the architecture.
Last updated August 2026.
