Is Web Scraping Legal? The Case Law That Decides It

Ask ten lawyers "is web scraping legal" and you'll get ten versions of "it depends." That's accurate, because no single statute answers the question. This article walks through the actual rulings that determine the answer: what courts decided, when, and why. If you're building a scraper and want to know where the legal lines sit, the case law tells you more than any generic disclaimer.

The short answer: it depends on what, how, and where

In the US, scraping publicly available, non-authenticated data is generally legal — courts have repeatedly declined to treat it as a computer crime. In the EU, the same activity carries more restriction, because data protection law applies regardless of whether the data was public. Is it legal to scrape data from a website, then? Usually yes, in principle, if the data is public and access wasn't gained by circumventing a technical barrier. The real risk comes almost entirely from how you scrape (bypassing logins, ignoring blocks) and what you collect (personal data, copyrighted text) — not from automated collection itself. For a scoring framework that turns this into a practical go/no-go checklist, see the developer's risk guide; this piece stays focused on the precedent.

Four separate laws decide the answer, not one

"Is scraping legal" is actually four independent legal questions layered on top of each other, and a project can pass three and still fail on the fourth.

First, anti-hacking law — in the US, the Computer Fraud and Abuse Act (CFAA). CFAA web scraping cases turn on whether access was "unauthorized," not whether scraping happened. Second, contract law: a website's Terms of Service is a private agreement, and a web scraping terms of service breach is a civil contract claim, separate from criminal computer-fraud law. Third, data protection law — GDPR web scraping personal data is regulated in the EU (and CCPA does similar work in California) regardless of whether the data sits behind a login. Fourth, copyright: scraping the data isn't the same as republishing or reproducing the underlying content, a separate infringement question. Each framework has its own tests, remedies, and case history — which is why the leading rulings below split their verdicts across them rather than delivering one clean answer.

The cases that actually set the rules

hiQ Labs v. LinkedIn (9th Circuit, 2019, remanded and settled 2022). hiQ scraped public LinkedIn profiles to sell workforce-analytics data. LinkedIn sent a cease-and-desist and blocked hiQ's IPs, then argued continued scraping violated the CFAA. The Ninth Circuit disagreed: scraping data that's publicly viewable without logging in isn't "accessing a computer without authorization" under the CFAA, because there's no gate to bypass. What gets left out: hiQ still lost ground on LinkedIn's contract-based claims once remanded, and the parties eventually settled with hiQ agreeing to stop scraping. The lesson is precise, not sweeping — the CFAA win didn't mean the scraping was consequence-free.

Meta v. Bright Data (N.D. Cal., summary judgment January 23, 2024). Meta sued Bright Data for scraping public, logged-out Facebook and Instagram data, alleging breach of contract and computer fraud claims. Judge Chen's summary judgment ruling largely favored Bright Data: claims tied to accessing logged-out, public pages were dismissed, while the court drew a sharper line around data gathered behind authentication. This case is now the clearest modern confirmation that scraping logged-out, public content sits on firmer legal ground than scraping anything requiring a login — a distinction every subsequent web scraping court case has had to address, including the wave of AI-training-data disputes still working through the courts. For the claim-by-claim breakdown of what Judge Chen dismissed versus allowed to proceed, see SociaVault's case analysis; for the jurisdictional contrast and exact filing date, see Cloro's compliance guide.

Van Buren v. United States (Supreme Court, 2021). Not a scraping case directly — a police officer misused database access he was authorized to have — but the Supreme Court's narrow reading of "exceeds authorized access" under the CFAA has become the backbone of every scraping defense since. It reinforced that CFAA liability requires bypassing a technical barrier, not just violating a policy or ToS clause. DataImpulse's synthesis ties Van Buren, hiQ, and Bright Data together well if you want a single reference.

Where it still gets you sued (or fined)

Winning on the CFAA doesn't mean you're clear everywhere. Four zones still carry real exposure. Scraping behind a login or paywall reintroduces the "unauthorized access" question the public-data cases avoided. Personal data — names, emails, profile details — triggers GDPR obligations in the EU and CCPA in California even when the data was sitting in plain view; is web scraping legal for personal data under GDPR depends on having a lawful basis for processing, not on public visibility. Reproducing copyrighted text, images, or structured databases verbatim can trigger infringement claims independent of how the data was collected. And hammering a server with aggressive, high-frequency requests can support a trespass-to-chattels or "computer damage" claim even without any unauthorized-access argument. Web scraping laws by country vary enough on all four points that a project legal in the US can still create liability in the EU or UK.

Does robots.txt actually matter legally?

Robots.txt is not a statute, and ignoring it isn't a crime by itself. Does robots.txt have legal force? Not directly — but courts and regulators have treated disregarding it as evidence of intent, used to show a scraper knew it wasn't welcome and proceeded anyway. That evidentiary weight can tip a close case, particularly in contract and bad-faith arguments, even though the file itself creates no independent cause of action.

Legal clarity doesn't solve the technical problem

Even a scraper that's fully compliant on all four fronts — public data, no login bypass, no personal data, no copyrighted reproduction — still fails in production if the target site blocks your IP, rate-limits your requests, or serves a blank shell because the content only renders client-side via JavaScript. That's a technical problem, not a legal one, and it's where a headless browser API earns its keep: rendering JS-heavy pages, rotating through blocks, and returning clean HTML, screenshots, or PDFs instead of empty divs. If your method is closer to screen-based automation than API calls, the screen scraping legal breakdown covers that distinction, and if you're actively getting blocked, this guide to spotting bot detection explains what's happening under the hood.

Being on solid legal ground is necessary, but it doesn't render a single page or retry a failed request. That's the infrastructure layer — headless rendering, proxy rotation, automatic retries — that turns a legally sound scraping plan into one that actually works at scale. Check the docs to see how a compliant scraping project gets built on a proper web scraping API, or compare plans on the pricing page.

Frequently Asked Questions

Is it illegal to scrape a website that has no robots.txt file?

No — the absence of a robots.txt file doesn't make scraping either legal or illegal by itself, since the file has no independent legal force. Legality still depends on the four separate frameworks: whether access required bypassing authentication, what the ToS says, whether personal data is involved, and whether content gets reproduced.

Can a company sue me just for scraping public data, even if I win the case?

Yes. Companies like LinkedIn and Meta have filed suit against scrapers of public data, and even when the scraper ultimately prevails — as in hiQ v. LinkedIn's CFAA outcome — defending the case costs time, legal fees, and often years of litigation. Winning the legal argument doesn't eliminate the cost of being sued.

Does using a proxy or headless browser to scrape make it more illegal?

No, the tool itself doesn't change the legal analysis — courts look at what data was accessed and how authorization was obtained, not which software made the request. Proxies and headless browsers are standard infrastructure; they only become relevant if used specifically to bypass a technical access control, which shifts the CFAA analysis.

Is it legal to scrape data for AI training purposes?

It depends on the same four frameworks, plus an added copyright layer specific to training-data reproduction that courts are actively litigating. Public, non-personal data scraped without bypassing authentication is on firmer ground than personal data or copyrighted creative works, but AI-training cases are one of the fastest-moving areas of scraping law right now.

What's the difference between scraping being illegal and just violating a website's Terms of Service?

Illegality typically means breaking a statute like the CFAA or GDPR, which can carry criminal or regulatory penalties. A web scraping terms of service breach is a civil contract dispute between the scraper and the site — it can result in being sued or banned, but it's a different legal category with different remedies than a criminal or regulatory violation.

Do I need a lawyer before starting a web scraping project?

For a hobby project scraping obviously public data, probably not. For a commercial project involving personal data, authenticated content, or another jurisdiction's data protection law, a short consult is worth the cost, since GDPR and CCPA penalties can far exceed a scraper's revenue if the lawful-basis question is wrong.