curl your own homepage. That is all ChatGPT sees.

Run this against your site right now:

curl -s https://yoursite.com | grep -o "<h1[^>]*>.*</h1>"

If nothing comes back, or you get an empty <div id="root">, then large parts of the internet cannot read your site. Not "reads it poorly". Cannot read it.

I do this on every site we take over, and the result surprises people often enough that it is worth writing down.

What the test is actually showing

curl does exactly one thing: it fetches HTML and stops. It does not run JavaScript. It does not wait for hydration. It does not call your API.

That is also what a large number of crawlers do.

Googlebot is the exception people think of, and it is genuinely good: it fetches, queues the page, and renders it with a headless browser later. Client rendered content usually gets indexed eventually.

The AI crawlers are a different story. As of now, the major ones (GPTBot, ClaudeBot, PerplexityBot, and friends) largely do not execute JavaScript. They fetch the HTML, take what is in it, and move on. Whatever your framework paints after the bundle loads is invisible to them.

So curl is a decent proxy for the floor: if your content is not in that response, assume a meaningful slice of automated readers never see it.

Why this got worse recently

For years the bet was reasonable. Google renders JS, Google is search, so client rendering was survivable.

Then a chunk of discovery moved to assistants. People ask ChatGPT for a recommendation instead of scrolling ten blue links. If the model cannot read your page, you are not in the answer, and there is no page two to be on.

For a marketing site this is the whole ballgame. For a small business it is worse, because the queries that matter ("web designers in X", "who does Y near me") are precisely the ones people now ask an assistant.

Three ways to check properly

1. Raw HTML, by word count.

curl -s https://yoursite.com | wc -c        # total bytes
curl -s https://yoursite.com | \
  sed 's/<script[^>]*>.*<\/script>//g' | \
  sed 's/<[^>]*>/ /g' | wc -w               # actual visible words

A marketing homepage with 40 words of real text in the raw HTML is a red flag. A React SPA often returns fewer than 10.

2. Compare raw against rendered.

// in DevTools console, on your live site
document.body.innerText.split(/\s+/).length

Compare that number to the wc -w above. A large gap is the amount of your site that only exists after JavaScript runs.

3. Ask with the right user agent.

curl -s -A "GPTBot" https://yoursite.com | grep -c "your-key-phrase"

Also worth confirming you are not blocking these crawlers by accident. Plenty of robots.txt files quietly disallow them because someone pasted a blocklist from a blog post in 2023:

User-agent: GPTBot
Disallow: /

That is a choice you can make deliberately. Making it by accident is expensive.

Fixing it

The fix is not "switch frameworks". It is "make sure the words are in the first response".

Next.js App Router. Server Components are the default, so you are mostly fine until someone adds "use client" at the top of a page to use one hook. Push "use client" down to the leaf that needs it instead of the page.

// page.tsx stays a server component
export default async function Page() {
  const data = await getData();
  return (
    <>
      <h1>{data.title}</h1>
      <p>{data.summary}</p>
      <InteractiveWidget />   {/* only this is "use client" */}
    </>
  );
}

The heading and copy are now in the HTML. The widget hydrates after. Both readers are served.

Vite, CRA, plain SPA. You need prerendering. vite-plugin-ssr / Vike, or a prerender step for known routes, or move the marketing pages to static HTML and keep the SPA for the app itself. The split is usually the honest answer: your dashboard does not need to be crawlable, your pricing page does.

Astro / Eleventy / Hugo. Already static. Check anyway, because a client only island can still swallow your main content if someone got enthusiastic.

Anything behind a cookie banner that blanks the page. Some consent implementations render an interstitial and nothing else until a click. Crawlers do not click.

The part people miss

Getting the HTML right is necessary but it is not sufficient. Assistants quote text. Give them text worth quoting.

• One <h1> that states what you do, in the words a customer would use, not a slogan.
• Answer the actual question on the page. If people ask "how much", a page that says "pricing tailored to your needs" gives the model nothing to repeat.
• Put facts in prose, not only in images or icon grids. Alt text is not a substitute for a sentence.
• Keep the important content above the fold in the DOM order, not just visually.

We rebuilt a manufacturer's site last year where the entire product range lived inside a JavaScript carousel. The raw HTML contained the company name and a cookie notice. Moving the product copy into server rendered markup was most of the work, and it was not a redesign, it was a text problem wearing a framework costume.

A 30 second checklist

# 1. Is there content at all?
curl -s https://yoursite.com | sed 's/<[^>]*>/ /g' | wc -w

# 2. Is the h1 there?
curl -s https://yoursite.com | grep -o "<h1[^>]*>[^<]*"

# 3. Are you blocking the crawlers?
curl -s https://yoursite.com/robots.txt

# 4. Does a key phrase survive?
curl -s https://yoursite.com | grep -c "what you actually sell"

Four commands. If all four look wrong, that is not an SEO problem to schedule for next quarter. It is the site not being readable, and it is usually a much smaller fix than a redesign.

Run it on your own site before you run it on a client's. I have been surprised more than once.