Do AI Assistants Actually Render Your JavaScript when Grounding? We Put It to the Test.

Our investigation into how AI assistants read web pages all began with a surprising inquiry from one of our clients. They had spent a fortune on a slick, modern front end where most of the meaningful content — the specs, the prices, the descriptions — was injected by JavaScript after the page loaded. Their question was simple: when someone asks ChatGPT or Gemini about our products, does the assistant actually see any of that?

What was our opinion? Honestly, we weren’t sure. Half of us assumed every modern AI runs a full browser by now; the other half suspected they cut corners. But “it depends” is not advice, and a guess is not a finding. So we ran our own experiment.

Why did we test this?

A growing share of the web is built with client-side frameworks — React, Vue, Angular and friends — where the raw HTML the server sends is little more than an empty shell. The text a human reads only appears after the browser downloads a script, runs it, and lets it paint content into the page.

For human visitors, none of this matters: their browser runs the JavaScript, so they see the finished page. The open question is whether an AI assistant does the same thing when a user pastes a link and asks it to summarize the page. If it does not — if it only reads the bare HTML — then for that assistant, a JavaScript-built page can be almost entirely blank.

We could have taken the vendors’ word for it, or trusted what each chatbot says about its own abilities. We didn’t, for one reason the test went on to confirm: what an assistant claims it did and what its servers actually did are not always the same thing.

The plan: Our own experiment

Our hypothesis was deliberately modest, because we had already been wrong once. Before the test, two confident predictions failed in turn: first “Gemini will render, because it’s Google,” then “nobody renders at all.” Only measurement settled it. So the plan was built to leave no room for opinion.

The core idea: hand each assistant a page where the truth was hidden behind executed JavaScript, then check the server logs to see who actually ran the code. If an assistant only reads raw HTML, it would confidently report a fake value. If it genuinely renders the page like a browser, it would report the real one — and the log would prove it.

We ran the test on a small website we run, on one production domain, against 12 AI assistants, one prompt each.

What is client-side rendering, and why can a page “lie” to a robot?

When your browser asks a server for a page, the server replies with HTML — the underlying text-and-tags blueprint. In an old-fashioned website, that HTML already contains all the visible content. In a modern JavaScript app, the HTML often contains only placeholders, and a separate script fills in the real content afterward by calling back to the server for data. This is client-side rendering, and it is everywhere: lazy-loaded sections, content behind tabs that hydrate on click, product data fetched from an API a half-second after load.

That gap creates a simple, powerful trick. You can put one value in the raw HTML and have the JavaScript quietly swap it for a different one. A reader that runs the script sees the second value; a reader that only reads the raw HTML sees the first. The page effectively shows two different “truths” depending on whether the reader executes code or just reads text. That is the mechanic at the heart of this test.

The decoy-versus-real mechanic, explained simply

We turned that gap into a trap, and the design is the most important thing to understand here.

• Each of the 12 assistants got its own secret, unguessable, never-linked URL — so the only way to reach a page was to be handed its exact link, and we could attribute every server hit to exactly one assistant.
• In the page’s raw HTML, the visible “internal reference number” was a decoy — a fake value, sitting right there in the source for anyone who only reads text.
• One line of external JavaScript replaced that decoy with the real number. Crucially, the real value lived only behind executed code: the script called a second endpoint, fetched the real number, and wrote it into the page. The real value appeared nowhere in the HTML and nowhere in the script’s own source. You could not find it by reading files as text. You had to run the program.
• We logged three things in real time, with full headers: the page request, the JavaScript file request, and the script’s follow-up call to that second endpoint.

The verdict logic falls out of this cleanly. If an assistant reported the decoy, it read raw HTML only. If it reported the real number, it executed the JavaScript — and a hit on that third endpoint in the server log is hard proof, because there is no way to reach that value by reading files as text. The machine had to have run the script.

Every page also carried a canary: a unique secret string that came back only if the page was genuinely fetched. That let us tell a real read from a confident hallucination.

The test setup in detail

Here is the full protocol, so anyone can replicate it.

The test deliberately mirrors real user behaviour, not the big background indexing crawlers. We pasted each secret URL directly into the assistant’s chat — by hand — and asked the same single question every time:

> “Summarize this page and report the internal reference number.”

This is the live, user-triggered “read this URL” path. It is what happens millions of times a day when someone drops a link into a chat window and asks for a summary. It is not the same thing as GPTBot, ClaudeBot and the other background crawlers that build training and search indexes; those may well behave differently, and that is a separate question.

For each assistant we then read three logs:

• Did the page get requested at all? (Was there even a fetch?)
• Did the JavaScript file get requested? (Did it look deeper than the HTML?)
• Did the follow-up endpoint get hit? (Did it actually execute the code?)

Then we compared the number the assistant reported in chat against what its servers had actually done. One site, twelve assistants, one prompt each, pasted by hand.

A note on scope, stated up front so it isn’t mistaken for an omission. This is a single-pass, live-fetch test: one prompt per assistant, captured at one moment in time. We deliberately did not run a second phase or chart the results over time, because the question here is binary — on a single user-triggered fetch, does the assistant run the JavaScript or not? — and a single clean pass answers it. A longitudinal study (does behaviour drift week to week?) is a worthwhile follow-up, but it is a different experiment.

The results

The headline finding is blunt: the entire US top tier reads raw HTML only. Not one major American assistant ran the JavaScript and used the result. The systems that did render were the four Chinese assistants and Mistral — a clean geographic split that nobody on our team predicted.

AI Assistant	Origin	Rendered the JavaScript?	What it reported
ChatGPT	US	No	Decoy (raw HTML)
Claude	US	No	Decoy (raw HTML)
Gemini	US	No — the surprise, despite being Google	Decoy (raw HTML)
Perplexity	US	No	Decoy; chat also claimed it “couldn’t access” the page
Meta AI	US	No	Decoy (raw HTML)
Microsoft Copilot	US	No — downloaded the script (via Diffbot) but never ran it	Decoy (raw HTML)
Grok (xAI)	US	Executed on one node, but ignored the result	Decoy (raw HTML)
DeepSeek	China	Yes	Real number
ERNIE (Baidu)	China	Yes	Real number
Qwen (Alibaba)	China	Yes	Real number
Kimi (Moonshot)	China	Yes	Real number
Mistral	Europe	Yes	Real number

Six of the most-used US assistants — ChatGPT, Claude, Gemini, Perplexity, Meta AI and Microsoft Copilot — reported the decoy. For most of them, the server saw only the page request and nothing more: they grabbed the HTML and stopped. Meanwhile DeepSeek, ERNIE, Qwen, Kimi and Mistral reported the real number, and the server log showed the full chain — HTML, then the script, then the execution-only endpoint.

The fetch-depth nuance

Lumping every non-renderer together would hide the most interesting detail, so look closer at how deep each system reached.

Most US systems requested only the HTML. They never even pulled the script file. But two cases sit in the grey zone between “reads text” and “runs code”:

• Microsoft Copilot was the only US system that even downloaded the external JavaScript file — it did so via the third-party scraper Diffbot. But downloading is not executing. There was no follow-up call, so the real value never materialized, and Copilot reported the decoy. Call it “fetched but not parsed.”
• Grok went one step further and then threw the result away. One node of its worldwide proxy swarm actually executed the JavaScript — the follow-up call appears in the log. And yet Grok’s answer still quoted the decoy. The machinery ran the script; the answer ignored what it found. The plumbing and the model were not talking to each other.

The canary proof

How do we know any of these reads were real and not the model bluffing? The canary. Every page that was actually fetched returned its exact, unique secret string — so these are genuine reads, not hallucinations dressed up as summaries. When an assistant reported the decoy, it wasn’t guessing; it was faithfully reporting what was in the raw HTML it had truly received.

Beyond the results: when the chatbot’s own story doesn’t match the logs

The most uncomfortable lesson has nothing to do with rendering. It is about trust.

1. Self-reports are unreliable. Perplexity told us in chat that it “couldn’t access” the page — while its own crawler had already fetched that exact page and received an HTTP 200. The assistant’s account of its own behaviour flatly contradicted the server log. If we had believed the chat window, we would have drawn the wrong conclusion.
2. The pipeline and the answer can disagree. Grok proved a system can run your JavaScript and still answer from the raw HTML. “Did a crawler touch my page?” and “did the answer use what the crawler found?” are two different questions.
3. Predictions are worthless here; measurement is everything. We were wrong twice before we measured — first betting on Gemini, then betting on nobody. Only the log settled it. That is a humility worth keeping when anyone tells you confidently how these systems “work.”

The throughline: when you want to know what an AI actually did with your page, trust the server log, never the chatbot’s description of itself.

The key takeaways

So what does this mean if you are trying to get found and cited by AI? On the live “open this URL” path, here is the practical reality.

If your important content is injected by JavaScript — client-side frameworks, lazy-loaded sections, content behind tabs that hydrate on load — then for ChatGPT, Gemini, Claude, Perplexity and Copilot, that content effectively does not exist. They will summarize and cite only your raw HTML. The assistants that would see your JavaScript content — the Chinese ones and Mistral — are, for most Western audiences, not the ones your readers are actually using. Rendering capability you can’t reach is no capability at all.

The path forward is not complicated:

1. Server-side render or pre-render anything you want an AI to read. If it isn’t in the HTML the server sends, assume the big assistants will miss it.
2. Put your key facts, claims and numbers in the raw HTML, high up the page — not buried behind a script that fires on load.
3. Use a canary when you genuinely need to prove a real fetch happened.
4. Trust the server log, never the chatbot’s claim about what it did.

This is, of course, a snapshot — one site, twelve assistants, one prompt each, at one point in time. Behaviour will change, vendors will ship new pipelines, and the background indexing crawlers may already render more than this user-facing path does. But the direction is clear, and it points the same way good SEO always has: toward content that is actually there, in the source, the moment the page is fetched.

Now it’s your turn. Have you tested whether AI assistants can see your JavaScript-rendered content? Have you watched a chatbot confidently describe a page it never really read? Share your experience — good or bad — and let’s open the conversation.

— Andre Alpar