· SPA, SSR and hydration
Reading Next.js hydration data without a browser
Next.js pages often ship their content as JSON in the first response. Where to find __NEXT_DATA__ and the App Router payload, and how to parse them safely.
View source first
Render a JavaScript page only when it needs it
Many pages still send their text in the first HTTP response. Check the raw HTML first; use a headless browser like Playwright only when the text appears after JavaScript runs.
Only Business, Enterprise and Custom sign-ups are reviewed before checkout. Failed fetches cost 0 credits.
The one-minute test
# 1. What does the server send?
curl -s https://example.com/ -o raw.html
# 2. Count <p> tags in the raw HTML
grep -oiE '<p[ >]' raw.html | wc -l
# 3. Look for an empty app shell
grep -oE '<div id="(root|app|__next)"></div>' raw.html
# Paragraphs present: a static fetch is enough.
# Empty shell, no paragraphs: the text is built
# in the browser. Check for embedded JSON next.The problem
Why JavaScript rendering scraping goes wrong
Teams either render every URL just in case, or they trust whatever the fetch returned. Both cost money, and two habits make both worse: ignoring data the page already ships, and waiting on the wrong signal.
Browser by default
Starting headless Chrome for every URL adds CPU, memory and cold starts to pages that already sent their text in plain HTML. The cost grows with every URL in the queue.
Silent blanks
A plain HTTP client gets a 200 with an empty <div id="root">. The pipeline logs success, and an LLM fills the gap with confident guesses about a page it never read.
Data you already had
Many client-rendered pages ship their content as JSON inside the HTML: hydration payloads, JSON-LD, Open Graph tags or a feed. Rendering them parses the same data the slow way.
Fragile wait logic
Rendered scrapes break on timing: waiting for network idle on a page that polls, or matching a container that exists before its text arrives. The result looks fine until it is empty.
The method
How to check for page content loaded by JavaScript
Four checks, cheapest first. Stop at the first one that gives you the text you need.
Step 1: Read the raw response
Fetch the URL without a browser, or use view-source:, not Inspect. If a sentence from the page is in the raw HTML, a static fetch can read it.
Step 2: Look for embedded data
Search the HTML for __NEXT_DATA__, __NUXT_DATA__, application/ld+json and og: tags. Content shipped as JSON can be parsed without running any script.
Step 3: Disable JavaScript once
Turn JavaScript off in DevTools and reload. What disappears is exactly the content that only exists after scripts run in the browser.
Step 4: Render only what is left
For pages that fail all three checks, use a headless browser and wait for the text itself, not for a container or for network idle.
Who this is for
Who needs JavaScript page to text, and why
The same question comes up from different sides: the people reading pages and the people building them.
Data engineers with empty results
Your HTTP client returns an app shell. Diagnose it in a minute and pick the cheapest fix that returns real text.
Teams paying for headless Chrome
Find out which URLs need a browser at all, and what Playwright, Puppeteer or a hosted browser API costs you per page.
Developers reading SPAs
Learn what CSR, SSR, SSG and streaming put in the first response, and where Next.js, Nuxt and Astro keep the data.
AI engineers cleaning context
Strip navigation and boilerplate from rendered DOM or raw HTML so the model reads the article, not the menu.
Pipelines that want structure
Read JSON-LD, Open Graph tags and RSS feeds that the page already sends, before you scrape the visible text.
Site owners invisible to AI tools
Check what GPTBot and ClaudeBot receive from your site, and fix client-only pages with SSR or prerendering.
Test it yourself
Fetch raw HTML, render in Playwright, or call a static fetch
The same URL three ways. Compare the outputs side by side before you decide which one your pipeline needs.
Raw HTML check
URL=https://example.com/
# Count <p> tags in the first response
curl -sL "$URL" | grep -oiE '<p[ >]' | wc -l
# Data shipped as JSON?
curl -sL "$URL" | grep -oE \
'__NEXT_DATA__|__NUXT_DATA__|application/ld\+json|og:title' \
| sort | uniq -cPlaywright
import { chromium } from "playwright";
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto("https://example.com/", {
waitUntil: "domcontentloaded",
});
// Wait for the text, not for the container
// (on your own page, use the element that holds the text)
await page.locator("body p").first().waitFor();
const text = await page.locator("body").innerText();
console.log(text.slice(0, 500));
await browser.close();Static fetch API
curl -X POST https://livewebdata.com/api/v1/contents \
-H "Authorization: Bearer $LIVE_WEB_DATA_API_KEY" \
-H "Content-Type: application/json" \
-d '{"urls": ["https://example.com/"], "renderFallback": true}'
# Text in the HTML: Markdown from a static fetch
# (statuses[0].strategy "fast").
# Needs JavaScript: rendered in a browser
# (strategy "browser"). Still empty: error tag
# CRAWL_NOT_RENDERED, with 0 credits charged.Live Web Data API does a static fetch by default. With renderFallback: true, only pages without usable content are rendered in a browser; the rest stay static.
Trade-offs
Static fetch vs headless browser: when each one wins
Neither is the right default for every URL. The useful question is which pages in your list actually need a browser.
Static fetch first
- Server-rendered pages
- Reads them in one HTTP request
- Client-only pages (SPA shell)
- Cannot read them; should flag them
- Main cost drivers
- Requests and bandwidth
- Timing logic
- None: one response
- Failure you must catch
- Empty shell treated as success
- Logins, clicks, forms
- Out of scope
Headless browser per URL
- Server-rendered pages
- Reads them, after starting a browser
- Client-only pages (SPA shell)
- Reads them once scripts finish
- Main cost drivers
- CPU, memory, cold starts, your ops time
- Timing logic
- Wait strategies per site
- Failure you must catch
- Timeouts and partial renders
- Logins, clicks, forms
- Possible
Hosted browser API
- Server-rendered pages
- Reads them, billed as a browser session
- Client-only pages (SPA shell)
- Reads them once scripts finish
- Main cost drivers
- Vendor pricing per session or minute
- Timing logic
- Wait strategies per site
- Failure you must catch
- Timeouts and partial renders
- Logins, clicks, forms
- Possible
SiteRender is run by Teneo Labs, which makes Live Web Data API, a static fetch that renders in a browser only on request. Rows describe tool categories, not named vendors, as checked against public docs in October 2026.
Guides
Guides to JavaScript pages, from raw HTML to text
Each guide answers one question end to end and closes with a test you can run on your own URL.
Do I need a headless browser? A one-minute test first
Four checks, cheapest first: raw HTML, embedded data, JavaScript off, and only then a browser. Run them on your own URL.
Render SPA to HTML: what each rendering mode sends first
What CSR, SSR, SSG, streaming and islands put in the first response, and where each framework hides the data.
Headless browser API explained: options, costs, wait logic
Playwright, Puppeteer, Lightpanda and hosted browser APIs compared on cost drivers, wait strategies and when each earns its keep.
Extract main content from HTML, raw or rendered
Readability, boilerplate removal, shadow DOM, iframes and lazy loading: getting the article out of the page, raw or rendered.
Structured data in HTML: read it before you render
JSON-LD, Open Graph, microdata, hydration JSON and feeds: the content a page already ships, and how to read it without a browser.
Can AI crawlers render JavaScript? What GPTBot sees
What GPTBot, ClaudeBot and other AI fetchers receive from a JavaScript-heavy site, and the framework fixes that make it readable.
FAQ
Questions about rendering JavaScript pages
How do I know if a website is rendered with JavaScript?
Compare view-source: with what you see on screen. If a sentence you can read on the page is missing from the raw HTML, it was added by JavaScript. Disabling JavaScript in DevTools and reloading gives the same answer visually.
Why does my scraper return an empty div instead of the page content?
The server sent an app shell, often <div id="root"></div>, and the text is added later by JavaScript that your HTTP client never runs. Before rendering, search the HTML for embedded JSON such as __NEXT_DATA__ or JSON-LD; the content is often already there.
Is a static fetch enough for JavaScript rendering scraping?
Often, yes. Article, documentation and product pages often send their text in the first response, and many client-rendered pages ship it as JSON. Use a browser only for what is left; the guides walk through the test.
How do I turn a JavaScript page to text without a browser?
Fetch the raw HTML and look for the text itself, then for embedded data such as __NEXT_DATA__, JSON-LD or a feed. If none of them holds the text, the page needs a headless browser.
Is Playwright or Puppeteer better for rendering pages?
For reading text both work. Playwright drives Chromium, Firefox and WebKit and has auto-waiting locators; Puppeteer is focused on Chrome and Firefox via CDP and WebDriver BiDi. The larger cost is running any browser, so filter out pages that do not need one first.
Does Live Web Data API render JavaScript?
No. It does a static fetch of public pages and returns Markdown, text or HTML. It is designed to return a CRAWL_NOT_RENDERED error when it finds no usable content, for example a page that needs JavaScript, and charges 0 credits for it, so your pipeline can route that URL to a browser.
Blog
Latest notes on rendering and raw HTML
Short, sourced posts on one rendering question each, with a test you can run on example.com.
· Does it need a browser?
Scraper returns empty HTML? Tell the four causes apart
An empty or near-empty response has four common causes: an app shell, a refusal page, compressed bytes and a soft 404. How to tell which one you have.
· Does it need a browser?
View source vs Inspect element: the one that matches curl
View source shows the HTML the server sent. Inspect shows the live DOM after scripts ran. Only one of them is what a plain HTTP fetch receives.
Test your URL before you render it
Check whether a static fetch is enough. Pages with no usable content return an error at no cost. Sign-up is for businesses only; only Business, Enterprise and Custom sign-ups are reviewed before checkout.