Your Answer Is on the Page. It Is Not in the HTML.

There is a gap between the page you look at and the page your server sends, and answer engines live on the wrong side of it. Everything below was found on this site, by running curl against our own URLs and reading what came back. None of it showed up in a browser, an audit tool or a Lighthouse run, because in every case the page looked perfect.

Google renders JavaScript before indexing. It has for years, which is why a client-rendered page can still rank. That habit is what makes this whole category of bug invisible: the search engine you have been optimising for is the one that papers over it.

Answer engines are a mixed population and many of them do not render. Some fetch the URL and read the response. Some read a cached copy taken by a crawler that did not execute scripts. Some are reading an index built by a third party. You do not control which, and you cannot detect it from analytics.

The safe assumption is the pessimistic one: if it is not in the response body, assume nothing quoted you. That single assumption turns an unknowable problem into a one-line check.

The most common one, and the most expensive. An FAQ accordion that mounts its answer only while the item is open. Closed, the DOM holds the question and nothing else.

So the served HTML of a ten-question FAQ is ten questions and zero answers. A model looking for a passage finds the question it was searching for, attached to nothing, and moves on to a site that answered it.

The fix is to collapse rather than unmount. Keep the answer mounted and hide it with max-height and opacity, so it is present for anything reading the page and tidy for the person looking at it. A native details element also keeps its text in the document.

FAQPage markup is easy to add and easy to forget about, and it drifts the moment the visible copy is edited by anyone who does not know the markup is there.

On our pricing page, the FAQPage described four questions. The number of those four that appeared anywhere on the page was zero. They had been written, marked up, and then replaced in the visible copy by different questions, and nothing connected the two.

That is a mismatch against Google's own requirement that structured data reflect content visible to the user. It is also a wasted move on its own terms, because the engines reading the DOM rather than the markup got nothing from it.

Generate both from one array. If the visible accordion and the schema are built from the same source, they cannot disagree, and a copy edit updates both at once. While you are there, count the FAQPage nodes on the URL: two is a duplicate-entity problem rather than two chances to be read.

An animated counter that rolls up to its value on scroll renders its starting value in the server HTML. Almost always that is zero.

Our homepage ran three of them. A visitor saw the figures count up. A crawler read the page as claiming "0+ leads handed off" and "$0k+ revenue credited back". The only audience that never saw the animation was the one quoting the page.

If a number matters enough to put on the page, render it as text and animate from it rather than to it. If it does not matter that much, the honest move is usually to delete it, which is what we did.

A dropdown navigation that mounts its panel on open puts none of those links in the server HTML. Most component libraries do this by default, and it is the right call for performance.

It is the wrong call for discovery if the nav is the only place a page is linked from. We had ten pages in that position: in the sitemap, reachable by a human through the menu, and carrying zero crawlable internal links from anywhere on the site.

A page with no internal link is discoverable and unranked. The fix is not to change the nav, it is to make the footer the real link graph and put everything that matters in it, or in a hub page that is itself linked.

The first four are technical. This one is editorial and it is the one most sites lose on, because nothing is broken.

A passage gets quoted when it can be lifted out of the page and still make sense. That means it starts with the answer, it is self-contained, and it stops. An answer that depends on the two paragraphs above it cannot be lifted, because the engine will not lift those as well.

The rule we write to: the question goes in an h2, the definition goes in the first sentence under it, and the qualification comes second. Never a build-up. If the passage cannot be pasted into a chat window and still read as an answer, it is not one.

The opposite failure, and rarer, but worth knowing because it is usually introduced by someone doing the right thing.

Our hero had a row of three proof points that rotated through twenty-one phrases. Rotating content is invisible to anything reading the page once, so the full set was added as a screen-reader list to make it accessible. That was the correct instinct and it produced a twenty-one item bulleted inventory in the HTML, sitting where a human reader sees one quiet line.

Anything parsing the page as text got a wall of bullets as the most structurally prominent thing near the top. We kept every word and made it a paragraph instead. Same content for a screen reader, no wall.

Run these against your own site before you buy anything that promises to measure AI visibility.

Worth adding and worth being honest about. It is a community proposal rather than an adopted standard, and no major answer engine has publicly committed to reading it. Anybody selling you an llms.txt strategy is ahead of the evidence.

The case for doing it anyway is modest and real. It costs an afternoon, it does nothing harmful, and writing one forces you to produce a clean index of what your site actually contains, which is a useful exercise whether or not a model ever reads the file. We generate ours from the same source as the site, so it cannot drift.

What it is not is a substitute for the HTML being right. A perfect llms.txt on a site whose answers unmount when closed is a tidy index pointing at pages that have nothing to quote. Fix the six above first. There is more on the measurement side in tracking AI visibility and on the file itself in how to write llms.txt.

Wondering how this differs from SEO? AIO versus SEO.