Is Web Scraping Legal? What the Rulings Actually Say
Four US cases decided the modern law of scraping and none of them turned on scraping. They turned on login, publicness, copying and circumvention.
Copy this line to your agent to read a page without holding the access risk yourself.
set up https://monid.ai/SKILL.md and use context.dev /web/scrape/markdown to read a public page as clean markdown
Four US cases decided the modern law of web scraping and not one of them turned on scraping as such. They turned on whether you were logged in, whether the data was public, whether you copied or merely read, and whether you got past a technical measure. Sort your own situation on those four axes and you will have a far better sense of your exposure than any article titled "is web scraping legal" can give you. This is not legal advice, and the cases below are US federal cases, cited so you can read them yourself. The tooling described later runs through Monid, the OpenRouter for agent tools.
Is web scraping legal?
Collecting publicly available information from websites is not, by itself, illegal in the United States, and US courts have repeatedly declined to treat it as computer intrusion. That sentence is the honest headline and it is also where most articles stop, which is the problem, because every one of those cases was decided on a narrower ground than "scraping is fine".
What the question is really asking
Nobody types this because they want a doctrine. They type it because they are about to point a program at somebody else's website and want to know what could go wrong. Four things could, and they are separate bodies of law that people routinely blend into one worry.
Computer intrusion. The Computer Fraud and Abuse Act, which is the criminal statute, and the one the early scraping cases were fought under.
Contract. The site's terms of service, which is a completely different question from intrusion and is where the cases have actually been turning lately.
Copyright. Whether you copied protected expression, as opposed to reading facts. Reproducing an article is not the same act as counting how many articles exist.
Circumvention. The Digital Millennium Copyright Act's anti-circumvention provisions, which are about getting past a technical protection measure and are their own separate hazard.
A scrape can be entirely clear on one axis and exposed on another. That is why "is it legal" has no useful answer and "which of these four am I touching" does.
And one that is not US law at all
If the pages contain personal data about people in the EU or the UK, GDPR applies regardless of what any American court held, and it attaches to the processing rather than to the collection method. Nothing in this article speaks to that, and it is the axis most likely to matter if your dataset is about people rather than about products or prices.
📖 See also Web Scraping Tools: Which Kind Do You Actually Need?
What did the courts actually decide?
Four rulings, in order, and what each one is and is not authority for.
hiQ Labs v. LinkedIn: the one everybody half remembers
hiQ scraped public LinkedIn profiles; LinkedIn sent a cease and desist; hiQ sued. The district court granted hiQ a preliminary injunction, 273 F.Supp.3d 1099 (N.D. Cal. 2017), and the Ninth Circuit affirmed, 31 F.4th 1180 (9th Cir. 2022), holding that accessing data the public can see without a password is unlikely to be access "without authorization" under the CFAA.
Then hiQ lost. On summary judgment, 639 F.Supp.3d 944 (N.D. Cal. 2022), the court found hiQ had breached LinkedIn's user agreement, because hiQ had accounts and turnkey access to those accounts. The case settled with hiQ enjoined.
That second half is missing from almost every explainer on this topic, and it is the more important half. hiQ won the criminal statute argument and lost the contract argument, which is the exact pattern the later cases repeat.
Meta Platforms v. Bright Data: what "use" means
Case No. 3:23-cv-00077-EMC (N.D. Cal.). On January 23, 2024, Judge Edward Chen granted Bright Data summary judgment on Meta's breach of contract claim. Meta's terms prohibit collecting data by automated means; Meta argued that anyone who visits Facebook is using it and is therefore bound.
The court disagreed on ordinary contract principles. Meta's terms govern "your use" of the products, and the court concluded that Bright Data did not use Facebook or Instagram when it engaged in public logged-off scraping. The court read the purpose of those provisions as preventing account holders from abusing the access their accounts give them, so an entity not using that access does not abuse it.
The takeaway is precise and it is not "terms of service do not matter". It is that terms bind users, and logging out can put you outside the category of user. Meta's tortious interference claim survived that ruling; Bright Data has since reported that Meta dismissed it. Farella Braun and Martel, who litigated the hiQ cases, published a detailed client alert on the decision.
X Corp. v. Bright Data: the same logic, again
In May 2024 a federal judge in the Northern District of California dismissed X's suit against Bright Data over scraping of public posts. Contemporaneous coverage reported the court's view that X was trying to have it both ways: claiming ownership of user content for the purpose of excluding scrapers while disclaiming it elsewhere. CNBC's report is the accessible summary.
Two platforms, two rulings, same year, same district, same result on logged-off collection of public posts. That is a pattern rather than an outlier.
Google v. SerpApi: a different statute entirely
This one is worth separating because it is not about the CFAA or terms of service. Google sued SerpApi on December 19, 2025, with claims under the DMCA's anti-circumvention provisions relating to SearchGuard, Google's anti-scraping system. SerpApi moved to dismiss in February.
On July 20, 2026, the Northern District of California granted that motion, and the split inside the ruling is the interesting part. The parts of Google's claims based on search results that did not contain copyrighted content were dismissed permanently. The parts involving results that did contain copyrighted content were dismissed with leave to amend, because Google had not alleged facts showing that SearchGuard was implemented and functioned with the authority of the copyright owner, which is what the statute requires.
SerpApi did not win everything, and the coverage that says otherwise is wrong. The court rejected SerpApi's argument that Google lacked standing, and found Google had alleged enough to support an inference that SerpApi circumvented SearchGuard. Discovery was stayed and Google was given 21 days to amend, so the case is live. Search Engine Land's report is the clearest summary.
What this case adds to the picture is the fourth axis. Anti-bot systems are increasingly framed as protection measures, and whether getting past one is circumvention is being litigated right now. It is the least settled of the four and the most likely to move. We wrote up the product-side implications in SERP APIs for AI Agents.
What makes a scrape risky rather than routine?
Four questions, in the order the cases suggest they matter.
Were you logged in?
This is the sharpest line in the current case law. An account creates a contract, and a contract creates obligations that survive whether or not the data is public. hiQ lost on this. Bright Data won twice because it was not logged in. If your pipeline authenticates, you are in a different legal position from one that does not, and it does not matter that the pages look identical.
Was the data behind a password?
Public means anyone can see it without credentials. Once a password wall is involved, the CFAA analysis changes materially, and the comfortable line of cases above stops applying to you. This is also the axis where "I made an account so I could see it" collapses the first question into this one.
Did you read, or did you copy?
Facts are not copyrightable; expression is. Counting how many products a competitor lists is a different act from republishing their product descriptions. Indexing an article so you can cite and link to it is different from reproducing its text. Most retrieval pipelines are on the safe side of this line and drift across it when somebody decides to store full article bodies and serve them.
Did you get past something built to stop you?
Rate limits, CAPTCHAs, bot fingerprinting. The DMCA question above is unresolved, and the practical advice while it is unresolved is conservative: solving a challenge designed to establish that you are human is a materially different act from sending a request from an address the site happens to accept. We are not going to pretend the line is crisp, because the litigation says it is not.
The axes as a table
| Axis | Lower risk | Higher risk | The case that shows it |
|---|---|---|---|
| Authentication | Logged out | Logged in with an account | hiQ II |
| Publicness | No password needed | Behind a login wall | hiQ, Ninth Circuit |
| Use of content | Facts, links, citations | Reproducing expression | Copyright generally |
| Access method | Ordinary requests | Defeating a protection measure | Google v. SerpApi, unresolved |
| Subject matter | Products, prices, listings | Personal data about EU or UK residents | GDPR, not US law |
Read down the left column and you have described most commercial scraping. Read down the right and you have described most of the litigation.
How do you scrape without inheriting someone else's risk?
Practically, three things, and none of them is a substitute for counsel.
For agents
Grab an API key at app.monid.ai, then paste this to your agent and hand it the key:
set up https://monid.ai/SKILL.md
It learns the whole discover, inspect, run workflow itself. More in the agent quickstart.
For humans
npm install -g @monid-ai/cli
monid keys add -k <your-api-key> -l main
Step 1. Prefer the official route when one exists
What it does. Removes three of the four axes at once. A licensed API is authorised access to data the provider chose to give you, under terms you agreed to on purpose.
The endpoints. Not an endpoint. Check whether the platform publishes an API before you build anything, and check what it costs, because "there is no API" is often "there is an API and it is expensive", which is a budget decision rather than a technical one.
What it costs. Usually more than scraping, and it buys a contract. Sometimes that is exactly what you are shopping for.
Step 2. Stay logged out, and keep it that way by design
What it does. Keeps you on the right side of the axis the cases turn on most often.
The endpoints. context.dev/web/scrape/markdown and context.dev/web/crawl fetch public pages without carrying credentials, which means there is no account to breach terms with.
The call.
monid run -p context.dev -e /web/scrape/markdown -w \
--query '{"url": "https://example.com/pricing", "useMainContentOnly": true}'
What comes back. Clean Markdown plus metadata: title, canonical URL, heading tree, Open Graph, and the page's JSON-LD when it publishes any. Structured data a site ships deliberately is the least contentious thing on the page, and it is often the data you wanted.
What it costs. A fraction of a cent per successful call, with failed and blocked fetches not billed. Current figures at monid.ai/tools.
Step 3. Store citations, not corpora
What it does. Keeps you on the read side of the copying axis.
The call. No endpoint. A design decision: keep the URL, the title, the publication date and the passage you actually need to quote, rather than every article body you ever fetched. Retrieval systems built this way are also better systems, because they cite instead of paraphrasing from memory.
What it costs. Nothing, and it removes the single most common way a well intentioned research pipeline turns into a copyright question. There is a worked version of a citation-first pipeline in Give Your AI Agent Live Web Context in One Call.
📖 See also Your Scraper Is Blocked: What Actually Gets Through in 2026
Give this to your agent
Set up https://monid.ai/SKILL.md, and then use Monid to read the public pricing page at this URL as markdown, keep the title, canonical url and published date, and quote only the pricing table.Which endpoint should I use for which job?
| Endpoint | What it does | Input | Output | Best for | Billing |
|---|---|---|---|---|---|
context.dev/web/scrape/markdown | Reads one public page | A URL | Markdown, metadata, JSON-LD | Logged-out reads of a known page | Per call |
context.dev/web/crawl | Walks a public site | Start URL, depth, page cap | Markdown per page | Public documentation and catalogues | Per page |
context.dev/web/search | Finds pages by query | A query | Ranked URLs with relevance | You do not have the URL yet | Per result |
pdl/v5/company/enrich | Returns a compiled company record | Domain or name | Firmographics with a confidence score | Company facts without scraping anyone | Per call |
Every row verified with monid inspect on 2026-08-25. The table gives billing shape rather than figures; current numbers are on monid.ai/tools.
The last row is the underused answer. A great deal of what people scrape is company information that somebody already compiled and licenses, and buying the record removes the access question entirely. The trade-offs there are in B2B Data Providers: Buy the Dataset or Call the API?.
When is scraping the wrong tool entirely?
Three cases where the answer is not a better scraper.
The platform sells the data. If there is a licensed feed and your objection is the price, that is a commercial disagreement, and losing it in court costs more than the licence. This is most of the financial data world and a good deal of the property and legal data world.
The data is about people and your users are in Europe. Personal data collection is a regulatory question before it is a technical one, and no amount of clean architecture makes it go away. Ask counsel before you build, not after.
A provider already shut this door. When a platform revokes access it usually stays revoked, and the sensible move is to migrate rather than to route around it. We wrote up one instance of that in the Proxycurl shutdown guide, and the lesson generalises.
And the disclosure this article owes you twice: you are reading Monid's blog, we sell per call access to tools including scraping ones, and that is a commercial interest in you concluding that scraping is workable. It is also why the honest framing here is a set of axes rather than a verdict. We are not your lawyers, nothing above is legal advice, and the citations are given in full precisely so you can take them to somebody who is.
Conclusion
"Is web scraping legal" has no answer because scraping is not one act. The cases that people cite for the permissive answer, hiQ and the two Bright Data rulings, all turned on a narrow finding about logged-out access to public pages, and hiQ itself went on to lose the part of its case that was about a contract. Anyone quoting the Ninth Circuit at you without mentioning 639 F.Supp.3d 944 is giving you half a case.
The more useful thing to carry away is the axis list. Logged in or logged out, public or gated, reading or copying, ordinary requests or defeating a protection measure. Those four decide almost everything, they are answerable about your own pipeline in about five minutes, and the answers are what a lawyer will ask you for anyway.
Free next step: run monid discover for the data you were about to go scrape and see whether somebody already licenses it. Discovery and inspection are free, and the cheapest legal risk is the one you did not need to take. Start at monid.ai.
FAQ
Is web scraping news legal?
Reading a publicly posted news page raises the same access questions as any other public page, but news adds a copyright dimension that product listings do not, because an article is protected expression rather than a set of facts. Indexing headlines, storing URLs and quoting a passage with attribution sits in very different territory from reproducing article bodies. If you are building a news pipeline, the practical guidance is in Web Scraping News Articles, and the rule of thumb is to store what you need to cite rather than everything you fetched.
Does violating robots.txt make scraping illegal?
No US court has held that ignoring robots.txt is by itself unlawful, and it is not a statute, a licence or a contract. It is a convention, and it is a strong signal about what the site operator wants, which matters both ethically and as evidence of notice if a dispute later arises. Treat it as something you should have a reason to disregard rather than as something with no weight, and note that a disallow you ignored will be exhibit one in any cease and desist that follows.
Do terms of service make web scraping illegal?
Terms of service create contract obligations, not criminal ones, and the recent rulings say those obligations attach to users of the service. That is why logged-out collection of public data survived summary judgment against both Meta and X, and why hiQ, which held accounts, lost on exactly this ground. The practical reading is that terms matter enormously if you have an account and much less if you genuinely never had one, which is a design decision you make before you build rather than a defence you construct afterwards.
Can a website sue me for scraping it?
Anyone can sue, and platforms do, so the useful question is what they would have to prove. On the current case law they would need a contract you were bound by, protected expression you copied, a protection measure you defeated, or access to something that was not public. If none of those describes your pipeline you are in the position Bright Data was in twice, and if one of them does you should be talking to a lawyer rather than reading a vendor blog.
Last updated August 2026.
