Is Scraping Websites Legal? A Developer's Risk Guide

The Short Answer

Scraping publicly available data is generally legal in the US. Is web scraping illegal? Not by default — the legal risk is a function of four variables: what you scrape, whether you authenticate to get it, how you copy or republish it, and where the people in your dataset live. A developer scraping logged-out, public product pages at a reasonable rate faces a very different risk profile than one scraping data behind a login wall while storing EU residents' personal information. This article maps those four risk layers — CFAA, contract/ToS, copyright, and privacy law — onto the technical decisions you make when you write the scraper.

The Case That Set the Rules: hiQ v. LinkedIn

Every "is scraping legal" conversation eventually cites hiQ Labs v. LinkedIn. LinkedIn argued that scraping public profile pages violated the Computer Fraud and Abuse Act (CFAA) — the federal law criminalizing "unauthorized access" to a computer system. The Ninth Circuit disagreed: because the pages were publicly viewable without logging in, there was no authentication barrier to bypass, so there was no "unauthorized access." That's the core of the ruling, and it's why most lawyers now say scraping logged-out, public pages doesn't trigger criminal CFAA liability.

But the story doesn't end there. On remand, hiQ ultimately lost — not on CFAA grounds, but on breach-of-contract theories tied to LinkedIn's User Agreement. As law firm analysis of the later rulings explains, the case pivoted from criminal exposure to civil contract risk, echoed in parallel litigation like Meta v. BrandTotal. CFAA liability and ToS liability are two different legal tracks, and hiQ shows both in one story.

Why Van Buren Matters Too

If hiQ narrowed the CFAA for scrapers, Van Buren v. United States locked that narrowing in at the Supreme Court level. The case didn't involve scraping — it concerned a police officer who misused database access he was otherwise entitled to — but the Court's interpretation of "exceeds authorized access" reshaped how courts read the CFAA generally. The justices held that the phrase means bypassing a technical restriction (like a login you're not supposed to have), not simply using access you do have for a disapproved purpose.

For scraper developers, Van Buren v. United States reinforces a clean rule: CFAA risk turns on authentication, not intent or policy violations. If there's no gate to bypass, there's no unauthorized access — regardless of what the site's Terms of Service say.

Four Legal Risk Layers Every Developer Should Check

Run these four checks before you ship your scraper:

1. CFAA / authentication bypass. Are you scraping logged-out, publicly accessible pages, or bypassing a login, CAPTCHA, or IP block? The former carries minimal CFAA risk per hiQ and Van Buren; the latter is where criminal exposure becomes real.

2. Terms of Service / breach of contract. Did you agree to a ToS (by creating an account, clicking "I agree," or accepting a click-wrap) that prohibits scraping? A violation is a civil contract claim, not a crime — but it can still mean lawsuits, damages, and injunctions, especially if you're logged in when you scrape.

3. Copyright on scraped content. Facts and raw data generally aren't copyrightable, but specific expression — article text, images, product descriptions — often is. Copyright risk grows sharply if you republish content verbatim rather than extracting facts or transforming the data.

4. Privacy law — GDPR/CCPA. Are you collecting personal data (names, emails, profile details) about identifiable people? If any are in the EU or California, GDPR and CCPA obligations can apply regardless of where your servers or company are based — "publicly posted" doesn't mean "fair game" under these frameworks.

Each layer is independent. You can pass the CFAA check and still lose on ToS. You can avoid a login wall and still violate GDPR by storing personal data without a lawful basis.

What Meta v. Bright Data Changed in 2024

The law kept moving in scrapers' favor for logged-out, public-page scraping through 2024. In Meta v. Bright Data, a federal court sided with Bright Data's scraping of publicly available Facebook and Instagram data, reinforcing that logged-out scraping arguments have real staying power in US courts — particularly where the scraper never agreed to a ToS by creating an account. As recent synthesis of scraping case law notes, this sits alongside growing CCPA activity and CNIL/GDPR enforcement fines targeting personal-data scraping in Europe.

The takeaway isn't "scraping is now fully legal" — it's that the gap between logged-out and logged-in scraping keeps widening. If your project touches authenticated data, read our guide on scraping data behind a login without breaking it before you write a line of code.

A Practical Compliance Checklist

Here's how to scrape legally without hiring outside counsel for every project:

• Check robots.txt and respect disallowed paths, even though it's not legally binding everywhere — it signals intent and courts notice.
• Avoid login walls unless you have explicit authorization or an API agreement with the site.
• Rate-limit requests so you're not degrading site performance — aggressive request patterns feed trespass-to-chattels arguments. See our rotating proxies vs. browser sessions framework for responsible request patterns.
• Strip or avoid collecting personal data you don't need — minimize what touches GDPR/CCPA scope.
• Don't republish copyrighted text, images, or layouts verbatim; extract facts, don't clone content.
• Stop immediately if you're blocked or receive a cease-and-desist — continuing after notice is a factor courts weigh heavily. Our post on spotting a bot-detection block covers how to recognize this before it escalates.

These are best practices, not legal loopholes — but following them meaningfully reduces exposure across all four risk layers.

How Infrastructure Choices Reduce Legal Risk

Legal risk and technical architecture aren't separate problems. A managed headless browser API that respects robots.txt, enforces sane rate limits, and handles sessions predictably is doing risk reduction, not just engineering. It's harder to accidentally hammer a server, retry through a block indefinitely, or scrape behind a login you weren't authorized to access when the infrastructure layer is built with those guardrails by default.

That's the practical case for a compliant scraping tool over a hand-rolled script that ignores 429s and keeps rotating IPs after a clear block signal. If you're comparing tooling options, our developer's category map of web scraping tools is a good next read.

Legal scraping is ultimately as much an infrastructure decision as a legal one. Check the Browsevra docs to see how the API handles rate limiting and session management, or review pricing if your team is ready to move off a fragile in-house scraper — start at browsevra.

Frequently Asked Questions

Is web scraping illegal in the United States?

No, scraping publicly available, logged-out web pages is generally legal in the US and doesn't violate the CFAA per hiQ v. LinkedIn and Van Buren. Risk increases sharply when you bypass authentication, violate a Terms of Service you agreed to, republish copyrighted content, or collect personal data covered by privacy law.

What happened in the hiQ v. LinkedIn case?

The Ninth Circuit ruled that hiQ's scraping of LinkedIn's public profile pages didn't violate the CFAA because there was no login or technical barrier to bypass. However, hiQ later lost on separate breach-of-contract grounds tied to LinkedIn's User Agreement, showing CFAA and ToS risk are distinct legal tracks.

Can a website's Terms of Service make scraping a crime?

No, a ToS violation is a civil contract issue, not a criminal one — it can lead to lawsuits or injunctions but not CFAA criminal charges on its own. Criminal CFAA liability generally requires bypassing an actual technical barrier like a login or IP block, per Van Buren v. United States.

Is it legal to scrape data if I have to log in first?

Logging in changes your risk profile substantially, since you likely agreed to a Terms of Service that may prohibit scraping, creating contract liability even without a CFAA violation. This is exactly the scenario where hiQ's later contract-based rulings apply, so login-gated scraping warrants extra caution.

Does GDPR apply to scraping public data about EU residents?

Yes, GDPR can apply to publicly posted personal data about EU residents regardless of where the scraper's company or servers are located. "Publicly available" doesn't exempt data from GDPR's lawful-basis requirements, and regulators including CNIL have issued enforcement fines over scraped personal data.

What should I do if I get a cease-and-desist letter for scraping?

Stop scraping the site immediately and route the letter to legal counsel before continuing any activity. Continuing to scrape after a clear block or written notice is a factor courts weigh heavily against defendants, turning a manageable dispute into a stronger claim for the website owner.