How to Scrape React/Vue Apps With a Headless Browser

Why Your Screenshot Is Blank (Or Your HTML Is Just a Root Div)

You point a headless browser at a React or Vue app, take a screenshot, and get a spinner — or the rendered HTML is just

with nothing inside. This is the most common failure when you scrape React Vue app headless browser setups without accounting for how single-page applications load.

The root cause: the browser's load event fires when the initial document, CSS, and JS bundles finish downloading — not when your app finishes running. A React or Vue SPA ships an almost-empty HTML shell; real content is built client-side after the JS bundle executes and the framework mounts components. Capture at load, and you get the shell, not the app. That's the blank screenshot headless browser problem: you waited for the network, but you needed to wait for JavaScript execution and DOM mutation, which happens afterward on its own schedule.

Server-rendered frameworks complicate this further. A Next.js page might arrive with real markup in the HTML (via SSR), but that markup is inert until React attaches event listeners and takes over — hydration. Capture too early and you get visually complete but functionally dead content, or a flash of pre-hydrated state before client-side data fetching overwrites it.

Why the Obvious Fixes Don't Work

The instinctive fix is to wait longer or wait for a "safer" event. Neither holds up in production.

domcontentloaded fires even earlier than load, before most JS has run, so it's rarely sufficient for CSR apps. load is better but still often fires before hydration completes, especially with code-split bundles, lazy-loaded components, or post-mount data fetching.

The next instinct is networkidle — wait until there's been no network activity for some window. This sounds robust but is explicitly discouraged beyond simple static pages, and current Playwright guidance confirms it: apps with polling, analytics beacons, websockets, or ad trackers may never go fully idle. Your scraper either times out waiting for a lull that never arrives, or hangs long enough to blow its time budget. This is the networkidle unreliable problem that trips up almost every team that tries it first.

Fixed sleep() calls are the third fallback, and they're brittle both ways. Sleep three seconds and a slow connection or heavy bundle still hasn't hydrated — partial capture. Sleep five seconds "to be safe" and you waste time on every fast page, multiplied across thousands of jobs. Worse, fixed delay scraping breaks silently the moment a target ships a bigger bundle or a slow third-party script — no error, just quietly wrong data.

What "Ready" Actually Means for React, Vue, and Next.js

The fix starts with understanding what actually happens at the framework level, so you know what to wait for instead of guessing a duration.

React mounts the component tree via createRoot or, for SSR pages, attaches to server-rendered markup via hydrateRoot. Until that call resolves and initial effects run, interactive and client-fetched content aren't in the DOM. Gatsby's explanation of hydration is a good primer on this mechanism, even outside the Gatsby context.

Vue has a comparable but distinct lifecycle: the app instance is created, then .mount() attaches it, triggering beforeMount, then mounted once the initial render commits. Vue's mount feels more synchronous than React's, but async components and Suspense-like patterns still mean "mounted" isn't always "fully populated with data."

Next.js adds another layer. With SSR, HTML arrives pre-rendered, but React still needs to hydrate it client-side before it's interactive. With the App Router and React Server Components, you may get streaming and progressive/partial hydration — different page "islands" become interactive at different times. Next.js hydration timing is genuinely hard to pin down with one generic rule, which is why a single wait strategy rarely transfers cleanly across React, Vue, and Next.js sites.

A Reliable Wait Strategy: Selectors, Custom Flags, and Bounded Fallbacks

Since no single browser event reliably signals "app is ready," treat readiness as something you detect explicitly, in priority order:

1. Wait for a specific content selector to be visible or attached. This is the most reliable general-purpose approach: waitForSelector in Playwright or Puppeteer, targeting an element that only exists once real data has rendered — a product title, a table row, a class the skeleton loader never uses. This is the practical answer to "wait for hydration before screenshot" for most sites.
2. Check a custom hydration-ready flag. If you control the target app, set window.__APP_READY__ = true (or a data-hydrated="true" attribute) once your root mount/hydrate call resolves, then poll for it with waitForFunction. This is the cleanest signal available and eliminates guesswork entirely.
3. Use MutationObserver-based readiness for unfamiliar pages. When you don't control the target, inject a script that watches the DOM via MutationObserver and resolves once mutations settle below a threshold rate — a more surgical version of what networkidle tries at the network layer, but applied to rendered content instead of requests.
4. Always cap with a bounded timeout. Every strategy above needs a hard maximum wait so a broken or unusually slow page fails fast with a clear error, rather than hanging a job queue.

Doing This at Scale Without Hand-Tuning Every Site

This works cleanly for one site but gets expensive fast across dozens or hundreds of target domains, each with different frameworks, loading states, and selectors that need discovering and re-validating whenever a target redesigns. Maintaining a custom Playwright or Puppeteer fleet with per-site wait logic becomes its own ongoing engineering project — a tradeoff covered in more detail in this cost comparison of self-hosted headless Chrome vs. a managed browser API.

A managed rendering API sidesteps this maintenance burden by exposing wait conditions as request parameters rather than code you babysit: pass a selector to wait for, a bounded delay, or an idle-with-timeout setting, and the API handles browser lifecycle, retries, and failure modes behind the scenes. Once you're capturing fully hydrated content, the next step is usually turning that HTML into structured fields — see this guide on building a field-mapping layer on top of a structured data extraction API.

Hand-rolled wait chains are brittle: they break when a site changes, they don't transfer between frameworks, and they multiply your maintenance surface with every new target domain. Browsevra's render API bakes in configurable wait conditions — selector, delay, and idle-with-timeout — so you get reliably hydrated screenshots, PDFs, and HTML without hand-tuning Playwright logic per site. Check the docs to see the wait parameters in action, or review pricing to evaluate usage-based plans. Learn more at browsevra.

Frequently Asked Questions

Why does my headless browser screenshot show a blank or loading page instead of the real content?

The browser is capturing the page before your JavaScript framework has finished mounting or hydrating. Events like load and domcontentloaded only track document and asset downloads, not React's hydrateRoot call or Vue's .mount() completing, so a capture taken right after those events fires often lands on the pre-render shell.

Should I use networkidle to wait for a React or Vue app to finish loading before scraping it?

Generally no — networkidle is discouraged for modern apps because polling, analytics beacons, and websocket connections can keep the network active indefinitely, causing timeouts. Selector-based or custom-flag-based waits are more reliable and are the currently recommended approach over relying on network quiet periods.

How is waiting for hydration different between React, Vue, and Next.js apps?

React hydrates by attaching to a root node via hydrateRoot, Vue mounts through a .mount() lifecycle with mounted firing after initial render, and Next.js can stream and hydrate progressively in "islands" when using React Server Components. Because the completion signal differs per framework, a single generic wait rule rarely works across all three.

Is a fixed delay (like sleep 3 seconds) good enough to wait for hydration?

No — fixed delays are unreliable because they're either too short on slow connections (partial captures) or wastefully long on fast ones, and they silently break when a target site's bundle size or third-party scripts change. A dynamic readiness check, like a selector or hydration flag, adapts automatically instead of guessing a duration.

How do I know what selector to wait for on a page I don't control?

Inspect the rendered DOM after full hydration and pick an element that only appears once real data is present — such as a specific product title, table row, or content container the loading skeleton never uses. If no such element is obvious, a MutationObserver-based readiness check that waits for DOM mutations to settle is a solid fallback.

Does server-side rendering (SSR) or Next.js streaming change how I should wait before capturing?

Yes — SSR delivers visible markup immediately, but that markup is inert until client-side hydration attaches event listeners and finishes any client data fetching. With Next.js streaming and React Server Components, different sections of the page can hydrate at different times, so waiting for a single top-level event isn't enough; you need a selector or flag tied to the specific content you need.