How to Get Cited by AI Search (Four Gates, In Order)
To get cited by AI search, four things have to be true in order: a crawler you allow, an answer that exists in the raw HTML, one self-contained passage that matches the question, and a fact inside it worth quoting. Miss any one and the rest is wasted. Here is how to check each gate today, with the 2026 numbers behind it.
TL;DR
- Allow the right bot. OpenAI's crawler documentation is explicit: sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers. GPTBot is the training crawler and is a separate decision.
- Ship the answer in HTML. Vercel and MERJ analyzed over 500 million GPTBot fetches and found zero JavaScript execution. Client-rendered content is invisible to most AI search.
- Write liftable passages. The Princeton and IIT Delhi GEO paper measured a 30 to 40 percent relative lift in visibility from adding statistics, quotations, and cited sources.
- Skip llms.txt. Google's John Mueller compared it to the meta keywords tag, and SE Ranking found no significant correlation across roughly 300,000 domains.
- Stop measuring clicks. Pew Research found users clicked through on 8 percent of searches with an AI Overview present versus 15 percent without, and 1 percent clicked a citation link. The citation is the placement, not the traffic.
AI search is a retrieval problem, not a ranking problem
Classic SEO competes at the URL level. You want your page to be the blue link. Generative engines do not work that way: they retrieve chunks of text from several sources, synthesize one answer, and attribute the sentences they used. The unit of competition is a passage, not a page.
That is why your rankings do not transfer. Brandlight's analysis found the overlap between top Google links and AI-cited sources has dropped from about 70 percent to under 20 percent. You can hold position one and appear in nothing.
Work the gates in order. Fixing your prose while a crawler is blocked is decorating a locked room.
Gate 1: allow the crawler that actually feeds the answer
The names matter more than people assume. OpenAI runs three agents and they do different jobs. OAI-SearchBot surfaces sites in ChatGPT's search features, and OpenAI states plainly that sites opted out of it will not appear in ChatGPT search answers. GPTBot is for training foundation models. ChatGPT-User handles a live request when someone asks ChatGPT to go look at a page, and it is not used to decide what appears in search.
The common mistake: a team decides it does not want its content training a model, blocks everything with an OpenAI-looking name, and quietly deletes itself from ChatGPT's search index. Those are two different decisions and you can take them separately.
A robots.txt that opts out of training while staying visible in AI search looks like this:
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
Check what you actually publish today before you change anything. curl -s https://yoursite.com/robots.txt takes two seconds, and a surprising share of blocks were added by a plugin or a security vendor that nobody remembers installing.
Gate 2: put the answer in the HTML, not in the JavaScript
This is the gate that quietly kills the most pages. Vercel and MERJ went through more than 500 million GPTBot fetches and found zero JavaScript execution. The crawlers download JavaScript files (ChatGPT in about 11.5 percent of requests, Claude in about 23.8 percent) and never run them. They read the HTML that came back on the first request and nothing else.
Gemini is the exception, because it uses Googlebot's rendering pipeline. AppleBot renders too. Everything else sees your unhydrated shell.
So a React or Vue app that ranks fine on Google can be a completely blank document to ChatGPT, Claude, and Perplexity. One command tells you which world you live in:
curl -s https://yoursite.com/your-page | grep -c "a sentence from your article"
If that returns 0, no amount of writing will help you. Server-render the page, statically generate it, or prerender it for crawlers. This blog is built exactly for that reason: the full article body ships in the initial HTML response with no client hydration, so the first byte a crawler reads already contains the answer.
Gate 3: write passages that survive being lifted out
Retrieval pulls a chunk. If your answer only makes sense with the two paragraphs above it, the chunk is useless on arrival.
Three edits do most of the work:
Make the H2 the question. Not "Our approach to pricing" but "What does it cost to run?" Question-shaped headings match query embeddings and give the chunker a clean boundary.
Answer in the first 40 to 80 words under it. Complete sentence, no pronouns pointing backwards, no "as we discussed above". Then expand underneath for the humans.
Put a hard fact in that answer. This is the part with a measurement behind it. The Princeton and IIT Delhi GEO study tested nine optimization methods across thousands of queries and found the winners were adding statistics, adding quotations, and citing sources, worth a 30 to 40 percent relative lift in position-adjusted word count over the unoptimized baseline. Keyword stuffing did not work. Writing more authoritatively did not work. Verifiable specifics did.
The test is mechanical. Copy any section out of your page, paste it into a chat with no other context, and ask whether it answers a real question. If it does not, the model has no reason to use it either.
Gate 4: be findable where the models already look
Retrieval does not start from nothing. Models reach for sources they have seen corroborated elsewhere, which is why third-party mentions do work that on-site edits cannot.
Two data points that seem to contradict each other, and both are useful. 5W's Q1 2026 audit found Wikipedia at 13.15 percent and Reddit at 11.97 percent of United States ChatGPT citations, more than a quarter between them. Evertune, looking at 200 million prompts, found the most-cited domain on any platform rarely exceeds 5 percent and that roughly 95 percent of citations are spread across thousands of domains.
Read together: a handful of aggregators own the generic queries, and the long tail is wide open for specific ones. You are not going to outrank Wikipedia on "what is generative engine optimization". You can absolutely own "how many pages does an AI crawler take per referral" if you are the person who published the number.
Practically, that means being present in the places that get retrieved: a real Reddit answer in your niche, documentation on your own domain, a YouTube description with the actual steps written out, and third-party posts that mention your name next to the claim you want attached to it. I keep the repeatable version of that workflow in the Viral Content Engine resource, and the OpusJake newsletter is where the new findings land first.
What to stop doing
llms.txt. It reads like it should work and it does not. John Mueller compared it to the meta keywords tag, Gary Illyes confirmed Google has no plans to support it, and SE Ranking found no statistically significant correlation across roughly 300,000 domains, with their model getting more accurate once the variable was removed. Ship it if you want. Do not count it as progress.
Chasing clicks as the KPI. Pew Research found an AI Overview cuts click-through from 15 percent to 8 percent, and only 1 percent of users click a citation link. If your success metric is sessions, AI search will look like a pure loss forever. The correct metric is share of answers: out of the 30 questions your buyer actually asks, how many name you.
Publishing more, faster. Volume without specifics is exactly the input these systems discount. The Princeton results are a warning here too. What moved the needle was verifiable content, not more of it. The same discipline I wrote about in reducing AI hallucinations applies to your own pages: an unsourced claim is a claim the model cannot safely repeat.
How to measure it without a new tool
Three checks, monthly, no vendor required.
- Log check. Group requests by user agent and count OAI-SearchBot, ClaudeBot, PerplexityBot, and Google-Extended hits against referrals from the same platforms. That ratio is your crawl-to-refer number, the metric Cloudflare Radar publishes. Cloudflare measured Anthropic's crawler at close to 71,000 page requests per referral in one late-June 2025 window, and noted a caveat worth remembering: traffic referred from a native assistant app often carries no Referer header, so referrals are undercounted.
- Prompt panel. Write 20 to 40 questions your buyer types. Run them monthly in ChatGPT, Claude, Perplexity, and Google AI Mode. Record whether you appear, and which page got cited. That spreadsheet is more useful than any rank tracker.
- Raw HTML check. After every deploy, curl one URL and grep for a sentence from the body. Rendering regressions are silent and they cost you every gate downstream.
The bottom line
Getting cited by AI search is unglamorous infrastructure work followed by better writing. Allow OAI-SearchBot and its equivalents. Serve the answer in HTML the crawler can read without running your app. Give each section a question for a heading and a self-contained answer with a real number and a source in it. Then get mentioned in the places the models already trust.
Do those four in order and you stop guessing. Skip the second one, which is what most teams do, and everything else you write is invisible to the systems you were writing it for.
How do you get cited by AI search?
Four things have to be true, in order. One, the right crawler is allowed in, which for ChatGPT means OAI-SearchBot and not GPTBot. Two, the answer exists in the raw HTML that crawler receives, because most AI crawlers do not execute JavaScript. Three, one self-contained passage on the page answers the exact question a person would type, without needing the three paragraphs above it for context. Four, that passage contains something worth quoting: a number, a date, a named source, a direct instruction. Miss any one of the four and nothing downstream matters. Most pages that fail are failing at gate two or gate three, and almost nobody checks either one.
Is GEO different from SEO?
The crawling and indexing half is the same work. The retrieval half is not. Classic SEO optimizes a page to rank as a blue link, so the unit of competition is a URL. Generative engine optimization competes at the passage level, because the model retrieves chunks and synthesizes an answer from several of them at once. That changes what a good page looks like: question-shaped headings, answers that stand alone inside a single section, and facts stated plainly instead of built up across a long argument. Brandlight's analysis found the overlap between top Google links and AI-cited sources has fallen from about 70 percent to under 20 percent, which is the practical reason you cannot assume your rankings carry over.
Does llms.txt help you get cited?
No, and the evidence is unusually clear for a question this new. No major provider reads it. Google's John Mueller compared llms.txt to the meta keywords tag, a self-declared signal nobody can verify, and Gary Illyes confirmed Google has no plans to support it. SE Ranking looked at roughly 300,000 domains and found no statistically significant correlation between having the file and how often a domain got cited, and reported that dropping llms.txt as a variable made their model more accurate. Studies of AI crawler logs find the file is almost never requested. Adding one costs you ten minutes, so it is not a disaster, but treat it as housekeeping and not as a strategy.
Do AI crawlers run JavaScript?
Mostly no, and this is the single most expensive technical mistake in AI search. Vercel and MERJ analyzed over 500 million GPTBot fetches and found zero JavaScript execution. ChatGPT and Claude crawlers do fetch JavaScript files (in roughly 11.5 and 23.8 percent of requests) and then never run them, so they only ever read the raw HTML. Google's Gemini is the exception because it rides Googlebot's rendering infrastructure, and AppleBot renders too. The result is a page that ranks first on Google and is a blank document to ChatGPT and Perplexity. Run curl on your own URL and read what comes back before you conclude anything else.
How do I know if AI search is sending me anything?
Check your server logs, not your analytics dashboard. Split requests by user agent (OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended) and compare crawl volume to referral volume, which is the crawl-to-refer ratio Cloudflare Radar publishes. Expect the ratio to look brutal: Cloudflare measured Anthropic's crawler at nearly 71,000 HTML page requests per referral in one late-June 2025 window, and the trend since has been more crawls and fewer referrals. Also note that some assistant apps send no Referer header at all, so real referrals are undercounted. Track citations directly instead: keep a list of 20 to 40 buyer questions and check monthly whether you appear in the answer.
OpusJake is Jake Schincariol's operating system for building with AI: agents, workflows, prompts, and the free resources behind them. Get the next move every week.