Server-Side Rendering for AI Search: No AI Crawler Renders JavaScript

Why No AI Crawler Renders JavaScript and What to Do About It

Vercel and MERJ tracked over 500 million GPTBot fetches and found something that should concern every site owner running a JavaScript framework: zero evidence of JavaScript execution. None. GPTBot downloads JavaScript files 11.5% of the time, but it never runs them.

Same story for ClaudeBot. Same for PerplexityBot. Same for Meta’s ExternalAgent. Same for Bytespider.

Only Googlebot renders JavaScript, using headless Chrome in a two-phase indexing system. Bing has partial JavaScript support. Every other AI crawler sees static HTML and nothing else. Rendering issues aren’t unique to AI crawlers. Here’s what breaks when Google hits heavy JavaScript.

If your website is built on React, Vue, or Angular with client-side rendering, your content does not exist in the AI crawlers’ view of the web. Not “performs poorly.” Not “partially visible.” Does not exist.

This is the most binary tactic in Generative Engine Optimization. You pass or you fail. I covered it alongside 14 other proven GEO tactics in the complete research breakdown. This article goes deeper on the technical details and exactly how to fix it.

The visibility impact is total, not gradual

Glenn Gabe published a case study demonstrating that client-side rendered content was completely invisible to ChatGPT, Perplexity, and Claude. Not partially indexed. Not underperforming. Invisible.

For sites built on React, Vue, or Angular with client-side rendering, every component rendered by JavaScript, every product description loaded via API call, every FAQ section injected dynamically, every comparison table built by a frontend framework is invisible to AI search. These are often the highest-value content elements on a site. And AI crawlers can’t see any of them.

This isn’t a performance optimization problem. It’s a binary visibility problem. Your site either serves content in the initial HTML response (visible) or it doesn’t (invisible). There’s no partial credit.

At Radiant Elephant, this is part of why our SEO process starts with technical SEO before content optimization. You cannot write your way around a rendering problem. If the crawler can’t read your content, no amount of statistics, expert quotes, or schema markup changes the outcome.

The double penalty for JavaScript-heavy sites

JavaScript-dependent sites face two compounding penalties:

Invisible content. JavaScript-rendered content isn’t seen by AI crawlers. All that carefully optimized text, those comparison tables, those FAQ sections, those product specifications? None of it reaches the AI.

Slower response time. Loading JavaScript frameworks adds latency to the initial server response, even for the HTML shell the crawler does receive. SE Ranking’s study of 129,000 domains found pages with First Contentful Paint under 0.4 seconds get cited 3x more than pages above 1.13 seconds. JavaScript frameworks push response times up while delivering content the crawler never processes.

Migrating to server-side rendering fixes both simultaneously. Pages become visible (content in the initial HTML) and faster (no JavaScript framework loading overhead).

Training crawlers vs. search crawlers: know the difference

Seventy-nine percent of top news sites block at least one AI training bot (BuzzStream/Hostinger data). But there’s a distinction most brands need to understand.

Training crawlers (GPTBot, ClaudeBot) ingest your content for future model training. Blocking them opts you out of future training data but doesn’t affect current AI search visibility.

Search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot) retrieve your content in real-time to generate answers to user queries. Blocking them removes you from AI search results entirely.

OpenAI updated its crawler architecture in December 2025 so that OAI-SearchBot and GPTBot now share information to avoid duplicate crawling. Blocking GPTBot only affects future training runs. Previously ingested content remains in the model. But blocking OAI-SearchBot means ChatGPT’s search feature can’t find your content when users ask questions your site could answer.

Most brands should allow search crawlers and make a deliberate decision about training crawlers based on content strategy and IP concerns.

How to diagnose the problem in 30 seconds

Open Chrome, go to Settings, find “Disable JavaScript” in DevTools, and navigate your own site.

What you see is what every AI crawler except Googlebot sees.

If your pages show a blank shell, a loading spinner, or empty content areas where text should be, you have a rendering problem. If your page source (View Source, not Inspect Element) shows <div id="root"></div> and a JavaScript bundle reference but no article text, no product descriptions, no FAQ content, you have a rendering problem.

Check server logs for GPTBot and PerplexityBot entries. Look at response sizes. A 200 response with a 2KB body means the crawler got the empty shell. A 200 response with a 15-50KB body means it has actual content.

How to fix it

Option 1: Server-side rendering migration. If your site is built on React, switch to Next.js and use getStaticProps or getServerSideProps for content pages. If Vue, switch to Nuxt.js with server-side rendering enabled. Both frameworks render the full page content on the server and deliver complete HTML in the initial response. AI crawlers see everything.

Option 2: Static site generation. For sites where content doesn’t change frequently (blogs, documentation, portfolios), generate static HTML files at build time using Next.js, Nuxt.js, Hugo, Astro, or Jekyll. Pages load in under 50ms from CDN edge and contain complete content in the HTML. The fastest and cleanest solution for content-focused sites.

Option 3: Pre-rendering service. If full SSR migration is too disruptive in the short term, implement a pre-rendering service (Prerender.io, Rendertron) that detects AI crawler user agents and serves them a pre-rendered HTML version. Users get the standard JavaScript experience. Crawlers get full HTML. This is a pragmatic bridge solution while planning a proper SSR migration.

Option 4: Replace JavaScript-dependent content elements. For WordPress or CMS-based sites where the core content is server-rendered but specific elements (FAQ accordions, pricing tables, tabbed content) rely on JavaScript, replace those elements with HTML/CSS alternatives. Use <details> and <summary> for accordions. Static HTML tables for pricing. Visible sections with headings instead of tabbed content that hides information behind JavaScript clicks.

After any fix, verify: View Source shows full page content. Disabling JavaScript still shows all critical text. Server logs show GPTBot receiving appropriately sized responses. Schema markup (JSON-LD) is present in the HTML source.

Five implementation examples

Example 1: React e-commerce site (headless Shopify)

Problem: Product descriptions, specs, reviews, and FAQ all load via JavaScript after initial page render. Disabling JavaScript shows a loading spinner and a product name from the <title> tag. Nothing else.

Fix: Switch from client-side rendering to server-side rendering in Next.js. Convert product pages to use getStaticProps with Incremental Static Regeneration (ISR), revalidating every hour. Product content (description, specs, reviews, FAQ) gets included in the initial HTML response. Real-time data (inventory levels) loads client-side after the static page renders. AI crawlers see all content. Users see real-time inventory.

Expected impact: Product pages go from completely invisible to fully visible. For products with strong organic rankings, AI citations can begin appearing within 1-2 weeks.

Example 2: Vue.js corporate website (single-page application)

Problem: Entire site is a Vue.js SPA. All navigation is client-side routed. Disabling JavaScript shows a blank page with a logo. The About, Services, Case Studies, and Contact pages all exist as Vue components that render only after the JavaScript bundle loads.

Fix: Migrate to Nuxt.js with generate mode for a corporate site with relatively static content. Nuxt pre-renders all pages as static HTML at build time. JavaScript dependency eliminated entirely for content delivery. Interactive Vue.js features preserved for users with JavaScript enabled. For a 20-30 page corporate site, migration typically takes 2-4 weeks.

Example 3: Angular documentation site (SaaS company)

Problem: 400+ documentation pages with content loaded dynamically from a CMS API after page load. Disabling JavaScript shows a header and empty content area. 400 pages of high-value technical content invisible to every AI crawler.

Fix option A: Angular Universal for server-side rendering within the existing Angular codebase. More complex than Next.js/Nuxt migrations but achievable.

Fix option B: Migrate documentation to a static site generator (Docusaurus, Hugo, Astro) that produces pure HTML. Documentation content changes infrequently, making SSG the cleanest solution. Removes JavaScript dependency entirely. Near-instant response times.

Recommended: Option B for documentation sites. Option A for application-like sites with frequently changing content.

Example 4: WordPress site with JavaScript content overlays

Problem: Core content is server-rendered (WordPress generates HTML by default) but FAQ accordions, pricing tables, comparison widgets, and testimonial sliders rely on JavaScript. Disabling JavaScript shows article content but FAQ sections collapse to nothing, pricing tables show empty placeholders, comparison widgets don’t render.

Fix: Replace FAQ accordions with <details> and <summary> HTML elements (content visible by default, JavaScript only adds animation). Replace pricing tables with static HTML tables. Replace comparison widgets with static HTML comparison tables. Replace testimonial sliders with static testimonial blocks. Replace tabbed content with visible sections separated by headings.

Key principle: Don’t remove all JavaScript. Just ensure all text content AI should cite is present in the initial HTML. Interactive enhancements can remain JavaScript-dependent.

Example 5: Headless CMS with React frontend (media company)

Problem: 500+ articles stored in Contentful. React frontend fetches content via API and renders client-side. View Source on any article shows <div id="root"></div> and JavaScript bundle references. Zero article content in the HTML.

Fix: Implement Next.js with getStaticProps for articles. Fetch content from Contentful at build time, include it in the HTML the server delivers. Use ISR with revalidation matching content update frequency (60 seconds for news, 24 hours for evergreen). The headless CMS remains the content source. The rendering moves from client-side to server-side.

Verification after implementation: View Source shows full article text. GPTBot server log entries show response sizes of 15-50KB instead of 2-5KB. AI citations for article content can begin appearing within days of crawlers re-indexing the now-visible pages.

Server-side rendering is a threshold requirement for GEO. Not a gradual optimization. Not a nice-to-have. A binary pass/fail. If your site fails, every other tactic in this cluster operates with zero content reaching AI systems. Fix this first. I mapped SSR alongside every proven GEO tactic in the full research review.

Server-Side Rendering for AI Search