Organic traffic in the AI era: SEO, AEO and GEO in practice

Over the past few months I've built and optimized three properties that are very different from one another:

• BluePaper — this studio: ~32 pages, a blog, case studies and per-vertical pages.
• CustoPorPrato (CPP), a pricing platform for food businesses, one of our own products, with a blog of 40+ articles and programmatic pages per niche. The case study is at CustoPorPrato.
• Índice Custo por Prato, a public price index: 496 thousand products, 410 supermarkets, 27 states, 6.1 million prices collected.

Three completely different scales, almost the same recipe, applied in layers. The most relevant numbers come from CustoPorPrato, a niche micro-SaaS that started as a tool for a few friends who were unhappy with what was out there:

• Organic traffic from Google: +884.9% — 1.6 thousand organic sessions in the period. The percentage is high because the baseline was small; I won't hide any numbers here.
• 103 thousand impressions and 3.2 thousand clicks in search, with an average position of 5.5 (Search Console).
• The site appeared in more than 1,000 AI answers tracked by Ahrefs: 548 in ChatGPT, 182 in Perplexity, 160 in Copilot, 64 in Gemini, 22 in Google's AI Overviews.
• 117 sessions came directly from AI assistants — a channel GA4 already classifies as its own category ("AI Assistant") and that didn't exist 3 months ago.

Are the numbers high? No. And the point here isn't big numbers — it's that you can still win ground with organic traffic while spending zero, especially if you're starting out with an idea and don't have (or don't want to spend) money on paid traffic.

It's also worth saying: I'm not an SEO specialist. This article documents what worked in practice, with the official sources linked at the end — and the mistakes I'd have avoided if I'd read the documentation first.

SEO, AEO and GEO: three consumers of the same content

The three terms get mixed up, so it's worth separating what each one means:

• SEO (Search Engine Optimization): the classic goal of ranking well on Google, Bing and search engines in general. The pattern that's existed for 20 years: search, click, visit.
• AEO (Answer Engine Optimization): being the answer, not just the link. Featured snippets, FAQ rich results, voice assistants. It requires content structured as a question with a direct answer.
• GEO (Generative Engine Optimization): being cited by LLMs — ChatGPT, Perplexity, Claude, AI Overviews. The term comes from an academic paper presented at KDD 2024 (citation at the end).

In practice, they're three consumers of the same structured content, each with its own delivery format. And Google's standard advice works for all of them: useful, indexable content with consistent structured data — the schema text has to be the same text that's visible on the page.

SEO: the basics done well (and where I got it wrong most)

The least glamorous part and the one with the highest return. Three mistakes I made and fixed:

1. Canonical per page, never in the layout. On Índice CPP, the product page accepts a ?uf= parameter to switch states. Without an explicit canonical per page, Google would treat each variation as a candidate URL: 27 URLs per product competing against each other in rankings. In Next.js, that means alternates.canonical in each route's generateMetadata() — never a fixed canonical inherited from the root layout, which would point every page to the same place.

2. Data-driven dynamic descriptions. The 400+ store pages started out with the same generic sentence, and Search Console flagged duplicate descriptions en masse. The fix was to build the description from the page's own data: store name, city, number of products, collection date. If the page is generated from a template, the description needs to be too — using the data that makes that page unique.

3. Explicit noindex on whatever shouldn't rank. Internal search pages, "not found" pages and infrastructure routes (300 MB+ of JSON) get an explicit noindex — via metadata or the X-Robots-Tag header, both documented in Google Search Central. With half a million pages, crawl budget stops being an abstract concept: every Googlebot visit to a useless route is a visit that didn't go to a page that matters.

The compound result: more than 60% of CPP's users today came from organic traffic — 41 new ones in the week I wrote this alone.

A sitemap with 496 thousand URLs

The sitemaps protocol caps each file at 50 thousand URLs or 50 MB uncompressed. With 496 thousand URLs, the solution was:

• A sitemap index pointing to shards: shard 0 holds the core pages (home, comparisons, 27 states, 410 stores); the following shards point to products in blocks of 40 thousand URLs.
• Curation: only products sold in 3+ states go into the sitemap. The rest remain indexable through internal links but aren't prioritized — a sitemap is a priority signal, not a complete inventory.
• A truthful lastmod (the real collection date): never a new Date() per deploy. Google uses lastmod when it's consistent and reliable, and starts ignoring it when the data "lies." An entire site with today's lastmod on every deploy is the definition of lying.
• priority and changefreq: I filled them in per page type, but Google ignores both (and Bing treats changefreq as a weak signal). I kept them because they cost nothing and other sitemap consumers may use them.

Programmatic SEO

The question that defines the structure: how do people search for what you offer?

• BluePaper: people looking for an agency want to know how to do it, how much it costs and how to hire. Pages per vertical served and per specific keyword — one template with different data is enough.
• CPP: pages per business type, per competitor, per ingredient and per state.
• Índice CPP: 27 states and 410 stores, 100% static at build time; product pages are generated on demand.

The rule that separates this from spam: a programmatic page needs real, unique data on each page — median price, coverage, comparison. A template with thin content is exactly what Google's spam policies call "scaled content abuse," and the penalty hits the whole domain, not just the page.

JSON-LD: the full vocabulary

Everything via schema.org, one application/ld+json block per schema — easier to validate and debug in the Rich Results Test:

• Organization + WebSite (with SearchAction) across the whole site.
• BreadcrumbList on every content page.
• FAQPage: since 2023 Google has restricted FAQ rich results to well-known government and health sites. The schema stays here because (1) it's valid for any consumer, not just Google, and (2) the rule still applies: the schema text is the same text visible on the page.
• Dataset: one of the most underrated schemas. Methodology (measurementTechnique), measured variables, temporal coverage, license, isAccessibleForFree — it's the schema that feeds Google Dataset Search and turns the index into a citable source.
• Product + AggregateOffer on product pages, with the real price range across stores.
• BlogPosting + HowTo on articles. HowTo, like FAQ, lost its rich result in 2023; I kept it as a semantic description.
• Person/ProfilePage on the author page. It's the technical side of E-E-A-T: content signed by a real person with a verifiable track record.

AEO: structuring content to be the answer

• Dynamic FAQ: on each state's page, the questions ("What's the cheapest supermarket in SP?") scale with the site and never go stale, because the answer is the data itself.
• Methodology page ("where the data comes from"): the GEO paper and the E-E-A-T guidelines point in the same direction — transparency about sources increases the chance of being treated as a source.
• Built-in glossary (CMV — cost of goods sold, markup, contribution margin, recipe costing sheet): terms defined in one sentence, a format answer engines pick up easily.
• Direct answer in the first paragraph, context afterward. It's counterintuitive — we're used to being strung along until the end to get the answer — but here it works better.

GEO: the new part (and what's convention, not standard)

Honest warning: much of what follows is emerging convention, not a recognized standard.

a) llms.txt — what it is and what it is NOT

It's a proposal (by Jeremy Howard, of Answer.AI) for a markdown index at the site root telling agents what the site is and where the key pages are; the llms-full.txt variant delivers the full content in a single file. Three facts the hype tends to leave out:

• It's a community convention, not a formal standard. There's no RFC and no IETF standardization.
• Google has stated in its official documentation that Search does not use llms.txt — neither for ranking nor for AI Overviews. No major LLM provider has publicly said it uses it as a citation signal (yet).
• Where it works today: coding agents and agentic browsing. Cursor, Claude Code, GitHub Copilot and similar tools fetch /llms.txt when pointed at documentation, and Chrome's Lighthouse gained an llms.txt audit in the "Agentic Browsing" category. Stripe, Vercel, Cloudflare, Resend and Anthropic itself publish the file for that audience — not for indexing.

I published it anyway: near-zero maintenance cost, zero downside (Google says it neither helps nor hurts) and upside for agents, which is where we're headed. But anyone selling llms.txt as "the new SEO" is lying.

b) A .md twin of every page

Every content page also exists as .md: clean content, no JS, no nav, no cookie banner. There's no secret to it — HTML for humans, markdown for agents, the same content in a fraction of the tokens. The strategy: explicit routes (append .md to the URL) and content negotiation via the Accept: text/markdown header on the same URL, with three safeguards to avoid creating duplicate content:

1. A Link: rel="canonical" header pointing to the HTML;
2. .md routes kept out of the sitemap — discovery happens through llms.txt and rel=alternate;
3. A correct Content-Type: text/markdown.

c) robots.txt with explicit groups for AI crawlers

GPTBot, OAI-SearchBot, ClaudeBot, Google-Extended, CCBot and others. A detail I learned the hard way: a crawler that finds a group with its exact name obeys only that group and ignores User-agent: *. So an explicit allowlist has to be deliberate, group by group — if you create a group for GPTBot with only Allow: /, none of the Disallow rules in the generic group apply to it anymore. Use Resend's or Cloudflare's robots.txt as a reference.

d) IndexNow

It notifies search engines the moment new content goes live, instead of waiting for a recrawl. Precision matters: IndexNow is adopted by Bing, Yandex, Seznam and Naver — Google does not take part in the protocol. For Google, the path is still a sitemap with an honest lastmod. And don't underestimate Bing: it powers a good share of ChatGPT's answers, and a fair amount of traffic came from there.

e) Proprietary data as a moat

The central thesis of GEO — and the only part I consider durable regardless of which convention wins: generative engines cite whoever has the data nobody else has. The GEO paper measured that including statistics and sources increases visibility in generated answers; an index with 6.1 million collected prices is the structural version of that. llms.txt may die, schema may change, but unique, verifiable data doesn't lose value.

Mistakes, limits and open items

• Attribution is fragile: +884.9% on a small baseline is less impressive than it looks, and I can't isolate how much came from each layer. What Ahrefs measures as "AI responses" is sampling, not a census.
• AggregateRating without real reviews is a temptation and a trap: self-serving review schema violates Google's structured data guidelines and can lead to a manual action. Don't do it.
• llms.txt is a bet, not a guarantee — I'm repeating it because it's the point where this article would sound most like a sales pitch if I didn't.

And finally: read Google's documentation. It would have saved me a lot of headaches.

References

1. GEO (academic paper): Aggarwal et al., "GEO: Generative Engine Optimization", KDD 2024 — arxiv.org/abs/2311.09735
2. llms.txt: proposal by Jeremy Howard (Answer.AI) — llmstxt.org
3. Google Search Central, guide to AI features in Search — developers.google.com/search/docs/appearance/ai-features
4. Google Search Central, structured data — developers.google.com/search/docs/appearance/structured-data and the announcement on reduced FAQ/HowTo rich results (Aug 2023) — developers.google.com/search/blog/2023/08/howto-faq-changes
5. Google Search Central, sitemaps (50 thousand URLs / 50 MB limit; position on lastmod/priority/changefreq) — developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap
6. Google Search Central, spam policies (scaled content abuse) — developers.google.com/search/docs/essentials/spam-policies
7. Sitemaps protocol — sitemaps.org/protocol.html
8. IndexNow (Bing, Yandex, Seznam, Naver) — indexnow.org

Frequently asked questions

What is the difference between SEO, AEO and GEO?

SEO (Search Engine Optimization) is ranking well on Google, Bing and search engines in general: the classic search, click, visit. AEO (Answer Engine Optimization) is being the answer, not just the link: featured snippets, FAQ rich results and voice assistants. GEO (Generative Engine Optimization) is getting cited by LLMs like ChatGPT, Perplexity, Claude and AI Overviews. In practice they are three consumers of the same structured content, and Google’s advice applies to all of them: useful, indexable content with consistent structured data.

Does llms.txt improve Google rankings?

No. Google stated in its official documentation that Search does not use llms.txt, neither for ranking nor for AI Overviews, and no major LLM provider has publicly said it uses it as a citation signal. Where it works today is with coding agents and agentic browsing: Cursor, Claude Code and GitHub Copilot fetch /llms.txt when pointed at documentation. It’s worth publishing because it costs almost nothing and has upside with agents, but anyone selling llms.txt as “the new SEO” is lying.

How do you build a sitemap for a site with hundreds of thousands of pages?

The sitemaps protocol caps each file at 50,000 URLs or 50 MB uncompressed. The solution is a sitemap index pointing to shards: one shard with the core pages and the rest in blocks. Beyond that: curation (only priority pages go in the sitemap), lastmod with the real date the content changed, never the deploy date, and knowing that Google ignores the priority and changefreq fields.

Does Google penalize programmatic SEO?

Only when it’s a template with thin content. That is what Google’s spam policies call scaled content abuse, and the penalty hits the whole domain. A legitimate programmatic page needs real, unique data per page: median price, coverage, a comparison. The question that defines the structure is how people search for what you offer, and each page has to answer it with data no other page has.

How do you show up in ChatGPT and other AI answers?

The durable part is having data nobody else has: the academic GEO paper measured that adding statistics and sources increases visibility in generated answers. Also helpful: a methodology page explaining where the data comes from, direct answers in the first paragraph, schema.org consistent with the visible text, and not blocking AI crawlers (GPTBot, ClaudeBot, PerplexityBot) in robots.txt. And don’t underestimate Bing: it feeds a large share of ChatGPT’s answers.