How to get cited by ChatGPT and AI search engines

The first reason you are not cited: the bots cannot read your site

There is a lot of talk about “GEO” (Generative Engine Optimization) as a brand new discipline. In our audits, the first problem is not new at all: the site simply cannot be read by the assistants’ crawlers. Check this before rewriting a single page, because no amount of editorial work makes up for a locked door.

Each vendor publishes its list of crawlers, and they do different jobs. Here is what OpenAI’s official documentation and Anthropic’s dedicated help page say, bot by bot:

OpenAI’s page states that a site blocking OAI-SearchBot will not be shown in ChatGPT search answers, and that a robots.txt change takes roughly 24 hours to be picked up. At Anthropic, all three bots honour robots.txt.

In practice, you can refuse training and accept search, or the other way round. That is a legitimate editorial choice. What is not legitimate is having the choice made for you without knowing it.

What we did: the Palmora Property case

Palmora Property is a real estate agency in Phuket we work with, with a site in six languages. The agreed strategy was clear: stay readable by AI assistants. The robots.txt file in the repository was clean and allowed everyone except private areas.

Three lessons from that episode:

• Check what is served, not what is written. One curl on the production URL showed the block. Reading the repository file would never have revealed it.
• The network layer can rewrite your files. A CDN, a web application firewall or an “AI bot protection” option enabled by default all act before your site does.
• The block had side effects. The non-standard lines of the managed block made Lighthouse report the robots.txt as invalid, which capped the SEO score at 92.

An honest caveat: the blocked list mostly targeted training crawlers, and OAI-SearchBot was not on it. So the site was not necessarily missing from every ChatGPT answer, but it was shut out for several vendors, against our will. Whenever the topic comes back, the first move is now to recheck that Cloudflare setting.

Answer straight away, and answer one question

Once the door is open, page shape matters. An assistant composing an answer extracts passages. A passage that makes sense on its own, lifted out of its page, has a better chance of being reused than a paragraph that assumes you read the three before it.

We apply two simple rules:

• The answer first. On this blog, every article opens with a direct answer of two to four sentences, displayed under the title. That is the fragment we want extracted. The rest of the article backs it up.
• One page, one question. That is the principle behind the knowledge base of Challenge Us, a group challenge app we built: 45 pages, each titled with a question in users’ own words, with a two or three sentence answer that has to stand on its own. The site refuses to build if two pages serve the same answer or if a title is not a question.

The same discipline helps people too: a reader in a hurry reads three lines and leaves with the answer.

Sourced and dated facts

The only serious academic study we know on the subject is “GEO: Generative Engine Optimization” (Aggarwal et al., presented at KDD 2024). The authors built a benchmark of 10,000 queries and tested nine ways of rewriting content. The three that worked best: adding quotations, adding statistics, citing sources. Keyword stuffing, on the other hand, brings almost nothing, and even does worse than the original text in the Perplexity test. They report these gains:

Read those figures for what they are. The visibility measured is the share of the answer’s text that comes from a source, weighted by position: not traffic, not revenue. The main experiments rely on a simulated engine (GPT-3.5-turbo fed with the top five Google results), and the authors themselves warn that the methods will need to adapt as engines evolve. It is solid evidence about direction, not a guaranteed recipe.

That direction matches what we do for other reasons anyway: on the sites whose content we write, no figure is published without a link to its primary source, actually opened. An assistant choosing between two pages has good reasons to prefer the one that says where its claims come from. And dating every page (publication date, update date, and for Challenge Us the app version in which the answer was checked) tells the reader, human or machine, what it is talking about.

Structured data, only for what is displayed

Structured data (JSON-LD) helps engines understand what a page is: an article, an organisation, a question and its answer. It is not a box to tick for AI engines. Google says so plainly in its documentation on AI features: there are no additional requirements to appear in AI Overviews, and no special file or markup to create.

The rule that matters is elsewhere. Google’s general guidelines forbid marking up content that readers cannot see. We turned that into a technical constraint: on the Challenge Us site, the displayed FAQ and the marked-up FAQ read the same list, so they cannot drift apart. A marked-up FAQ that does not exist on screen is a penalty you build yourself.

How to measure without fooling yourself

This is the part “GEO” offers skim over the most. There is no Search Console for assistants. Here is what we do, and nothing more:

1. A list of target queries, typed by hand into ChatGPT, Claude and Perplexity on a fixed schedule, noting whether the site is cited and which page. It is manual, answers vary from one session to the next, and that is precisely why we repeat the measurement instead of drawing conclusions from one try.
2. Server or CDN logs, filtered on the bot names (OAI-SearchBot, GPTBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot). They tell you whether the bots come and which pages they read. They do not tell you whether you are cited.
3. Visits from assistants in your analytics, when the referrer is passed. That is a lower bound: many citations produce no click at all.

What we do not do: promise a number of citations. Nobody controls what a model decides to cite. An agency that guarantees it is selling an impression, not a result.

Where to start

Tackle the work in this order:

The llms.txt file comes after that, and we have detailed what you can really expect from it. On a multilingual site, add a hreflang check: an assistant does not cite a page it cannot find in the right language, and the most common mistakes are easy to spot.

These checks are part of our SEO audit, and the ongoing work of our SEO and GEO engine. The overall method is laid out in what an SEO agency should deliver in the age of AI.

Frequently asked questions

Sources

1. OpenAI’s official documentation (new tab)developers.openai.com
2. Anthropic’s dedicated help page (new tab)support.claude.com
3. “GEO: Generative Engine Optimization” (new tab)arxiv.org
4. Its documentation on AI features (new tab)developers.google.com
5. Google’s general guidelines (new tab)developers.google.com