Logbook — Our Homepage Was 29 KB of Nothing

8 min read

Our Homepage Was 29 KB of Nothing

One useSearchParams() under a root <Suspense> turned every page of our Next.js site into an empty shell for crawlers. How I found it, fixed it and guarded it.

  • nextjs
  • seo
  • react

I was auditing the landing page at Refer from two directions at once: reading the code, and fetching the live site the way a crawler would. They gave me opposite answers.

The code said the content was server-rendered: Next.js App Router, React components, nothing exotic. The live site said the <body> was empty.

Both were telling the truth about what they showed. So I looked at exactly what the server sent: 29 KB of HTML, and zero characters of text in the <body>. This is what was in there:

<body class="antialiased min-h-dvh flex flex-col">
  <div class="flex-1"></div>
  <template data-dgst="BAILOUT_TO_CLIENT_SIDE_RENDERING"></template>
  <script src="/_next/static/chunks/…"></script>
  …
</body>

Every one of the site's 23 routes looked like that. In the browser, every one of them looked perfect.

Who actually reads your HTML

A browser downloads that empty shell, runs the JavaScript and fills the page in. Unless you're looking for it, you never see the gap. Googlebot renders JavaScript too, in a second pass, so Google eventually saw our content.

AI crawlers are a different story. When Vercel studied the crawler traffic on its network at the end of 2024, none of the major AI crawlers rendered JavaScript. The exceptions were Google's Gemini, which rides on Googlebot, and Applebot. GPTBot and ClaudeBot even download some of the JavaScript files, but they don't run them. For them, our site was a <title> and nothing else.

Play with it. The page is made up, but the three ways of serving it are the three stages our site went through:

What does this reader get?
Who is reading
Root layout
Scroll reveals animate
Fetches the HTML and doesn't run JavaScript, like GPTBot and ClaudeBot. What the server sends is all it gets.
acme.dev
<div class="flex-1"></div>: no text in the HTML at all.
0 of 403 characters visible · 403 not in the HTML
What the server sends (the same for every reader)
<body class="antialiased min-h-dvh flex flex-col">
  <div class="flex-1"></div>
  <template data-dgst="BAILOUT_TO_CLIENT_SIDE_RENDERING"></template>
  <script src="/_next/static/chunks/…"></script>
  …
A made-up page served three ways. The counts are characters of this mock page, not of any real site.

One hook, every page

The cause was eight lines in the root layout:

// app/layout.tsx, before
<body className="antialiased min-h-dvh flex flex-col">
  <Suspense fallback={<div className="flex-1" />}>
    <UTMProvider>
      <div className="flex-1">{children}</div>
      <MobileFooter />
    </UTMProvider>
  </Suspense>
</body>

UTMProvider reads the campaign parameters from the URL (?utm_source=…) so the signup form can tell where a visitor came from. To read them, it called useSearchParams().

The Next.js docs explain what that does: on a statically rendered route, useSearchParams() makes the client component tree up to the closest <Suspense> boundary render on the client. You're supposed to wrap the small component that needs the URL in a <Suspense> so only that piece waits for the browser. (With no <Suspense> above it at all, next build fails on a static page, so wrapping everything is the quickest way to make the error go away.) Ours wrapped {children}. So the server rendered the fallback, an empty <div className="flex-1" />, for every page, and left the rest to JavaScript.

Nothing complained. Lint, type-check and tests passed, the browser looked fine, and CI deployed to production on every merge. The <Suspense> was already in the layout when the site moved into our monorepo at the end of April. The fix shipped on September 2.

The fix didn't need a <Suspense> at all. UTM capture only matters in the browser anyway (it saves the parameters for the signup form to read on submit), so the provider now reads window.location in the effect it already had:

// hooks/useUTM.tsx, after (simplified)
useEffect(() => {
  // Not useSearchParams(): here, around {children}, it took every page out of SSR.
  const params = new URLSearchParams(window.location.search);
  saveUTM(params);
}, [pathname]);

Bonus bug: every page said it was the home page

While I was in the layout I found its metadata had alternates: { canonical: "/" }. Child routes inherit the layout's metadata unless they override it, and none of ours did. So /for-organizations told search engines: "I'm a duplicate of the home page, index that one instead." Every page did.

They couldn't override it, either. Every page was a client component ("use client"), and metadata can only be exported from server components. Why were they client components? None of the four marketing pages used a single React hook. They were "use client" only because they used Framer Motion's motion.div for their scroll animations.

So I took the animation out of the pages. The hero copy became a plain, unanimated <h1> and <p>. Sections that already lived in their own client components kept their animation there. What was left went into one small client component, <Reveal>, that wraps server-rendered children. With no motion code left in them, the pages became server components, each with its own title and canonical URL. The root layout lost its canonical. (That one is now a rule on my own site, too: never put a canonical in the root layout.)

Text in the <body>, measured in the prerendered HTML:

PageBeforeAfter
/02,258
/about01,875
/for-organizations05,393
/for-your-career03,637
/terms (and 4 more legal pages)03,053

Measuring is where I got it wrong (twice)

The first time I counted the bailout markers left after the fix, grep -c said one per page. But grep -c counts lines, and Next.js ships the whole document on one line. grep -o … | wc -l counts occurrences: two.

Both turned out to be harmless. One was Vercel's SpeedInsights, which wraps itself in its own <Suspense>. The other was a video player loaded with dynamic(…, { ssr: false }), on purpose, because it only works in a browser. Neither wraps page content. The page you're reading has one too: Vercel's <Analytics /> also calls useSearchParams(), but inside its own <Suspense fallback={null}> that wraps nothing else. That's the pattern done right. So the number to watch is how much text is in the body, not how many markers there are.

The second mistake came a few minutes later and was funnier: my script reported that 107% of the home page's text was hidden. Nested elements were being counted once for each wrapper. Merging the ranges first fixed it, and the real number was worse than I expected.

79% of the home page was opacity:0

The scroll animations looked like this:

<motion.div initial={{ opacity: 0, y: 20 }} whileInView={{ opacity: 1, y: 0 }}>

Framer Motion renders the initial state into the server HTML as an inline style, style="opacity:0;transform:translateY(20px)". Only JavaScript, once the element scrolls into view, takes it away. Once the server was rendering the pages again, 79% of the home page's text sat inside opacity:0 elements. Only 476 characters were visible without JavaScript.

My first explanation, which even made it into a code comment, was that Googlebot never scrolls, so whileInView never fires for it. That's only half right. Google's docs do say Search doesn't interact with the page, so no scrolling or clicking. But Google's John Mueller has explained that Googlebot renders with a very tall viewport instead, so content that appears when it enters the viewport does get loaded. Googlebot probably saw that text.

The argument that holds is simpler: the HTML itself says most of the page is invisible. That's what every reader that doesn't run our JavaScript gets: a visitor on a slow phone whose bundle is still downloading, someone with JavaScript turned off, a crawler that applies styles but doesn't run scripts. Whether a given crawler ignores inline styles is up to that crawler. I'd rather not bet on it when the fix costs nothing:

'use client';
import { motion } from 'motion/react'; // Framer Motion's current package
 
export function Reveal({ children }: { children: React.ReactNode }) {
  return (
    <motion.div initial={{ y: 20 }} whileInView={{ y: 0 }} viewport={{ once: true }} transition={{ duration: 0.6 }}>
      {children}
    </motion.div>
  );
}

It still rises into place for real visitors. It just never starts invisible. After the change, 95–97% of the text on the home page and /for-organizations is visible in the HTML, and 100% on the other pages. What's left is the back face of a flip card, which should be hidden.

A guard that has never failed isn't a guard

This bug shipped because nothing was looking. So CI now runs verify:seo right after next build, and the production deploy waits for it. It reads the prerendered HTML and fails when:

  • a page has less body text than its floor, with an error that names the likely cause (a provider calling useSearchParams() above {children});
  • more than 15% of a page's text sits inside opacity:0;
  • a page doesn't have exactly one <h1>;
  • the JSON-LD is missing or doesn't parse;
  • a canonical points anywhere but the page itself;
  • a route exists on disk but isn't in the manifest that generates the sitemap.

Then I broke the build output on purpose to watch the guard fail. A check you've never seen fail might be checking nothing.

Check your own site

This is the same measurement as a script you can point at any URL. Node 18 or later, no dependencies:

// crawler-eye.mjs: what does a crawler that doesn't run JavaScript get?
// Usage: node crawler-eye.mjs https://example.com
const res = await fetch(process.argv[2], { headers: { 'user-agent': 'crawler-eye/1.0' } });
const html = await res.text();
const body = html.match(/<body[^>]*>([\s\S]*)<\/body>/)?.[1] ?? '';
const text = (s) =>
  s.replace(/<(script|style)[\s\S]*?<\/\1>/g, '').replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim();
 
// Text inside opacity:0 elements. Merge the ranges so nested wrappers count once.
const ranges = [];
for (const m of body.matchAll(/<(\w+)[^>]*opacity:\s*0[;"][^>]*>/g)) {
  const tags = new RegExp(`</?${m[1]}\\b[^>]*>`, 'g');
  tags.lastIndex = m.index + m[0].length;
  for (let depth = 1, t; (t = tags.exec(body)); ) {
    depth += t[0].startsWith('</') ? -1 : 1;
    if (depth === 0) { ranges.push([m.index, tags.lastIndex]); break; }
  }
}
let hidden = 0, end = -1;
for (const [s, e] of ranges.sort((a, b) => a[0] - b[0])) {
  if (e <= end) continue;
  hidden += text(body.slice(Math.max(s, end), e)).length;
  end = e;
}
 
const total = text(body).length;
const bailouts = html.split('BAILOUT_TO_CLIENT_SIDE_RENDERING').length - 1;
console.log(`${(Buffer.byteLength(html) / 1024).toFixed(0)} KB of HTML`);
console.log(`${total} characters of body text, ${hidden} of them inside opacity:0`);
console.log(`${bailouts} client-side rendering bailout marker(s)`);

Our home page today:

75 KB of HTML
2307 characters of body text, 105 of them inside opacity:0
2 client-side rendering bailout marker(s)

The 105 is the flip card, and the two markers are the harmless ones. It's a regex, not a real HTML parser, so treat it as a smoke test.

The same bug, three times

One last thing. The same layout, with the same bailout, had been copy-pasted into two other codebases, one of them a dead repo. Part of my first audit was done on that dead copy, by mistake. A layout is one of the most copied files in a Next.js project, which makes it the worst place to hide a bug. Archive the dead copies, or someone will fix the wrong one.

The short version:

  • Look at what the server sends (curl it) and measure the text in <body>, not the bailout markers.
  • Never call useSearchParams() in a component that wraps {children} in a layout. Read window.location in an effect, or put a tight <Suspense> around the one component that needs it.
  • Don't set a canonical in the root layout.
  • Animate transform, not opacity, for scroll reveals.
  • Put the check in CI, and break it once on purpose.

Quick answers

Why does useSearchParams() make my Next.js page render on the client? On a statically rendered route, the server doesn't know the query string, so Next.js renders the client components up to the nearest <Suspense> boundary in the browser instead. If that boundary wraps {children} in a layout, it's every page. (With no boundary at all, the build fails.)

What does BAILOUT_TO_CLIENT_SIDE_RENDERING in my HTML mean? It marks a <Suspense> boundary that rendered its fallback on the server and will render its content in the browser. One isn't necessarily a problem; what matters is what it wraps. Check how much text is in your <body>.

Do AI crawlers run JavaScript? In Vercel's December 2024 analysis of traffic on its network, none of the major AI crawlers did. Gemini, which uses Googlebot, and Applebot were the exceptions. Server-render anything you want them to read.

Does Googlebot scroll? No. Google says Search doesn't interact with the page. According to Google's John Mueller, Googlebot renders with a very tall viewport instead, so content that appears when it enters the viewport still loads.

Is text with opacity:0 bad for SEO? Search engines don't publish how they treat it. What's certain is that the HTML marks that text as invisible to anything that doesn't run your JavaScript. Animating transform instead costs nothing.

Sources