SEO Rank Media
Future ready, human first.
Schedule A Call

The Two Arenas of AI Search Optimization and the Process I Use to Win Both

Caleb Turner on March 19, 2026

If you want to win in AI search, you have to win two distinct, but related arenas. Getting retrieved and getting preferred.

I don’t treat AI visibility like a vanity ranking game. I care more about whether a brand is consistently pulled into the right conversations and preferred for high intent prompts.

The research supports that approach. The 2023 GEO study introduced Generative Engine Optimization and reported visibility gains of up to 40% from the right optimization methods.

The E-GEO: A Testbed for Generative Engine Optimization in E-Commerce study found that intent match, factuality, differentiation, and scannable formatting consistently improved ranking outcomes.

I look at AI search as a two-step system.

First, your brand has to get retrieved. That means your site, brand, or page has to make it into the possible answer set. Then, once you are in the set, you have to get preferred. That second step is the re-ranking layer, where the model decides which option is the clearest fit for the user’s prompt. The 2025 E-GEO paper formalizes that same retrieval-plus-ranking framework.

That is why I don’t separate off-page SEO from on-page AI optimization. Off-page authority, mentions, citations, links, and digital PR help you get pulled in. Clear, well-structured, evidence backed content helps you get chosen once you are there.

LLM difference

I do not chase vanity rankings. Ranking for a head term like IT managed services might look good in a dashboard, but it tells me almost nothing about whether I am visible when a real buyer is actually close to choosing a provider. 

People looking to purchase are not stopping at short category phrases anymore. They are asking long, detailed, constraint-heavy buying questions like:

 “We run a 12-location healthcare group in Dallas and need a HIPAA-compliant IT provider that can manage endpoints, harden Microsoft 365, support audits, and help with compliance documentation. Who should we talk to?” 

“Which Dallas managed IT companies can provide backup and disaster recovery with a one-hour recovery time objective, a 15-minute recovery point objective, and 24/7 incident response?” 

“We need a managed cybersecurity partner for a multi-site medical practice that can handle SIEM, endpoint detection, phishing training, and policy documentation. Who actually does all of that well?” 

This shift is not a theory anymore. Amazon says shoppers are already using Rufus to type natural-language questions, compare options, and ask granular product questions, while Bain found that 42% of large language model users already use these platforms for shopping recommendations.

In other words, I care less about whether I “rank #1” for a trophy keyword and more about whether search engines and large language models consistently understand that my brand is a strong answer for the high-intent situations my market actually cares about. That is what matters.

Query journey

The process I use to increase AI visibility

My process starts with query selection, not page editing.

First, I identify the money prompts. These are the commercial, comparison-driven, bottom-of-funnel queries that reflect how people actually ask for help today. Then I expand those into related prompt variations so I can see the full intent map around the topic. 

After that, I compare my page against competing pages and measure which page is most semantically aligned to the query. If my page is weak, I do not guess why. I fix the relevance gap.

Then I optimize at the chunk level.

Modern AI systems often retrieve and reuse sections, not entire pages. That is why I built an internal workflow that pulls strong content chunks, scores them against the target query, rewrites the best chunk into cleaner formats, and then re-ranks the outputs to identify the version most likely to be extracted and preferred.

In practice, the best version is usually the one that is easiest to scan, easiest to trust, and easiest to quote.

What I like about the 2025 E-GEO study is that it pushes the conversation beyond hype.

The researchers tested more than 7,000 realistic product queries, evaluated 15 common rewriting heuristics, and found that the best results came from a stable pattern rather than a gimmick. The content that rose tended to align closely with user intent, preserve factual accuracy, clearly differentiate itself, and present information in a format that is easy for the model to process.

I have had conversations in person and even on my YouTube channel with industry leading colleagues like Nick Eubanks, Ross Simmonds, and Charles Floate, and even when the tactics vary, the same fundamentals keep coming up: build authority beyond your site, understand how people really search, and publish content that is clear enough to be reused in answers.

That is a big reason I believe AI visibility is not a trick. It is a discipline.

GEO optimization

You need distribution. You need credibility. You need answer-ready content. And you need to stop measuring success with outdated vanity metrics that were built for a different search era.

AI visibility is not about gaming one prompt or chasing one keyword. It is about increasing the probability that your brand gets retrieved for the right conversations and is preferred when the model compares options.

Parent/head terms still matter as category and entity anchors for retrieval, internal linking, and query reformulation, but they do not deserve to be your primary measurement system for AI-era revenue visibility. 

The KPI should be whether your brand is retrieved and preferred across the high-intent prompt set that real buyers actually use.

That means winning both arenas: off-page presence strong enough to get you into the set, and on-page structure strong enough to move you to the top.

That’s the process I use, and it’s the process I keep seeing validated by research, by testing, and by conversations with other people deep in this space. If you focus on those fundamentals, you stop chasing vanity and start building visibility that actually compounds. Schedule an introductory call with me today to discuss how we can do this for your brand.

What the 2025 GEO Study Reveals About Ranking in AI Answers (and How I Apply It)

Caleb Turner on January 27, 2026

TL;DR

I built a custom internal tool inspired by that study to help me (and my clients) turn messy, long-form information into “AI-ready” answer blocks that are more likely to be pulled and preferred in AI results. We’re actively expanding that tool to make it more comprehensive, and it’s already being used to guide client content.

I’ve spent the last couple of years watching the same pattern play out across industries:

  • Some brands get mentioned in AI answers… but show up as the third or fourth recommendation.
  • Others get pulled sometimes… but inconsistently.
  • And a few dominate because their content is the easiest to trust, quote, and rank.

That’s why I pay attention when researchers publish something that goes beyond opinions and actually tests what works.

The research team from MIT and Columbia University ran the study (E-GEO: A Testbed for Generative Engine Optimization in E-Commerce) to measure how answer engine rankings change based on how content is written. They used thousands of real-world shopping-style queries, rewrote product descriptions using different approaches, and tracked whether the rewritten versions moved up or down in AI rankings.

What I’m sharing below is the practical version of what matters, and how you can apply it whether you sell products or services.

The two-step reality: getting pulled vs getting preferred

RAG process

Summary of RAG process

Most brands only think about one part of the problem.

Step 1: Retrieval (getting pulled in)

This is whether your page, brand, or product even gets included as a possible answer.

Step 2: Re-ranking (getting preferred)

This is where the AI decides what’s “best,” “second best,” and so on.

The 2025 study focused heavily on that second step: once options are in the set, what makes one rise to the top?

If you’re investing in content for AI visibility, you want both:

● content that reliably gets pulled
● content that reliably gets ranked as the preferred option

What the researchers actually did (in plain English)

Reddit queries matched to Amazon Listings

Reddit queries matched to Amazon Listings

Here’s the simple version of the test:

  1. They took 7,000+ real “what should I buy?” style posts from Reddit.
  2. They matched each query to the 10 most relevant Amazon product listings using a semantic similarity method (basically, “meaning match,” not just keyword match).
  3. They rewrote product descriptions using different prompt styles.
  4. They measured whether the rewritten descriptions moved up in the AI’s ranking.
  5. They then used a second model to iteratively improve the rewrite prompts until the rewrites performed better.

The purpose wasn’t to find a trick. It was to find patterns that hold up repeatedly across many queries and products.

That’s exactly what brands need right now, repeatable rules, not hype.

The most important practical takeaway

ranking in ChatGPT

ranking in ChatGPT

AI rankings reward content that makes it easy to confidently choose and easy to quote.

That’s why the winning patterns consistently included:

1) Clear intent match

The best performing rewrites aligned tightly with what the user actually asked for, especially long, conversational queries with constraints.

Not “Knife set for kitchen.”

More like:
“Premium durable knife set with minimal upkeep needed.”

That shift matters because it directly mirrors the user’s real goal.

2) Factual grounding

One of the most consistent themes was factuality. Keep claims accurate, avoid embellishment, and preserve what can be supported.

In real life, that means:

● don’t guess
● don’t inflate benefits
● don’t claim “best” without support

3) Clear differentiation (your “why us” without fluff)

Competitive positioning mattered, especially when it was expressed as concrete differences, not generic marketing language.

Examples:

● materials, specs, certifications
● what’s included vs not included
● warranty terms
● durability, maintenance requirements
● constraints the product/service is best for (and not best for)

4) Evidence signals

When you can back something up with data or reputable references, do it. Evidence helps the AI system “trust” the content and reuse it.

For service businesses, evidence can be:

● licensing and certifications
● documented process steps
● before/after metrics
● review volume and rating
● pricing ranges and what drives them

5) Scannable formatting

This one is huge, and it matches what I’ve seen in the wild. Content that’s easy to scan is easier for AI systems to lift into answers.

Headings, bullets, short blocks, ranges, and direct definitions beat long paragraphs every time.

What didn’t work as well

GEO and AEO Heuristics that did not work well

GEO and AEO Heuristics that did not work well

A lot of “internet advice” about AI visibility leans into tone or style:

● “Write like an ad”
● “Be super persuasive”
● “Tell a story”
● “Sound authoritative”

The study showed that these approaches can be inconsistent, and in some cases can even hurt performance if they reduce factual clarity or drift away from the user’s intent.

The takeaway: style is secondary; structure and usefulness come first.

The custom tool I built (inspired by the study)

meta optimizer for GEO and AEO

meta optimizer for GEO and AEO

This study didn’t just confirm what I suspected, it gave me a structure I could build around.

So I built an internal GPT-powered re-ranking workflow inspired by the study’s re-ranking logic.

Here’s what it does in practical terms:

What my tool does today

  1. I feed it a target query (example: “How much does carpet cleaning cost in Northern Virginia?”).
  2. It ingests multiple content “chunks” pulled from top-performing pages on Google.
  3. It ranks those chunks based on which one most directly and completely answers the query.
  4. It rewrites the best chunk into multiple output formats, especially:

○ a tight “answer block”
○ a highly scannable version with headings and bullets

  1. It re-ranks the outputs and recommends the version most likely to be extractable and preferred in AI answers.

In short, it helps us consistently produce content that is accurate, aligned with intent, and formatted for extraction, which is exactly what the study suggests is repeatably effective.

How we’re expanding it

Right now we’re expanding the workflow to be more iterative, and mathematical about which content is most optimized. This will result in:

● content that more accurately reflects what AI engines reward
● stronger evidence handling
● consistency checks
● broader content formats

How we’re using it for clients

We’re already using this tool in our client work to:

● upgrade existing pages into AI-ready “answer-first” structures
● produce scannable sections that AI engines can quote cleanly
● reduce fluff while increasing proof and clarity
● align content with how people actually ask questions in AI tools

This is one of the ways we’ve been able to move faster while staying grounded and factual, because the tool forces discipline around intent, structure, and evidence.

Final thoughts

ai seo

ai seo

The 2025 MIT + Columbia E-GEO study supports something I’ve been emphasizing for a while:

If you want to rank well in AI answers, your content has to do more than “sound good.” It has to be the clearest match to the user’s intent, backed by facts, and formatted in a way that’s easy to extract and trust.

That’s why I built a tool around this, and why we’re expanding it and using it actively in client content.

If you want your site to show up more often and be preferred in AI answers, request my free AI visibility checklist. I’ll review your site and tell you:

● what’s preventing AI engines from pulling your pages
● what’s keeping you from being ranked as the “top” recommendation
● which pages to fix first for the fastest impact

You can implement the changes yourself, or my team at SEO Rank Media can handle it for you.

Answer Engine Optimization (AEO) Explained: What It Is, What Services an AEO Agency Provides, and Who Needs It

Caleb Turner on January 14, 2026

Answer Engine Optimization (AEO)—sometimes called AI search optimization or Generative Engine Optimization—helps your brand, products and services appear in AI‑generated answers by structuring content, data and authority so that AI systems can understand and quote it. For companies seeking an AEO agency in Los Angeles, this article explains what AEO is, what services agencies provide and who benefits from them.

Key Takeaways

 

    • AEO ensures your brand appears in AI‑generated answers and voice assistants.

    • It complements SEO and paid ads by helping you get chosen, not just found.

    • AEO agencies align content, entities and structured data for AI comprehension.

    • B2B SaaS, professional services, high‑consideration products and local services benefit most.

    • When hiring an AEO agency in Los Angeles, ask about outcomes, schema, content structure and measurement.

AI search is changing how people find products and services. More than 60 percent of Google searches now end without a click, and voice or conversational queries will account for roughly half of all queries in the coming years. When a buyer types “best onboarding software” into an AI‑powered platform such as Perplexity or ChatGPT, the answer engine doesn’t show a list of blue links, it gives a curated comparison. If your brand isn’t mentioned in that answer, you don’t even exist to that buyer.

That’s where answer engine optimization comes in. AEO prepares your content and entities so AI systems understand them, cite them and include them in answers. This explainer will demystify AEO in plain language, compare it to traditional SEO and paid ads, outline what an AEO agency does, describe who needs this service, and provide practical guidance on hiring the right partner.

What “AEO” stands for (in plain English)

AEO acronym for answer engine optimization

 

In simplest terms, AEO is the practice of making your brand, products and information easy for AI-powered search engines to understand and quote. Unlike traditional search engine optimization (SEO), which focuses on getting your website to rank in the top positions of a results page, AEO increases the probability that your information is selected and presented inside AI‑generated answers. Instead of only being concerned with clicks, AEO also aims to get your brand mentioned or cited by answer engines.

AEO in practice

When someone asks an AI engine, “Which accounting software should a Los Angeles–based startup use?” the model quickly scans its knowledge sources, decides which entities and products are relevant, and then constructs a concise answer. A company optimized for AEO will have:

 

    • Clear entities—The company name, product names, modules and key use cases are defined consistently across its website and third‑party profiles.

 

    • Structured data—Pages include schema markup (e.g., FAQ, Product, Organization, HowTo) that tells AI engines exactly what each piece of content is about.

 

    • Answer‑ready content—Content is formatted using question‑and‑answer style summaries, concise bullet lists and headings that align with the questions buyers ask.

Because of these signals, the answer engine can quickly extract the relevant information and highlight the brand in its summary. That is the essence of AEO: help AI help your customer find you.

AEO vs. SEO vs. paid ads: what’s different

SEO, AEO and paid advertising all aim to put your business in front of potential customers, but they operate differently. The table below summarizes the core distinctions.

Approach Goal Target surfaces Optimization techniques Typical outcome
Search Engine Optimization (SEO) Improve rankings in traditional search results Google/Bing organic results Keyword research, on‑page optimization, link building, technical SEO Drives clicks to your website to read longer‑form content
Answer Engine Optimization (AEO) Get included in AI‑generated answers and voice responses AI Overviews, ChatGPT Search, Perplexity, Google Bard, Claude, etc. Entity modeling, schema markup, concise Q&A content, authority building Delivers direct brand mentions, citations, and/or comparisons.
Paid Advertising Purchase visibility in search results or social platforms Google Ads, LinkedIn, Facebook, display networks Bidding on keywords, targeting audiences, ad creative

Generates traffic immediately but disappears when you stop paying

 

AEO doesn’t replace SEO or advertising, it complements them. Traditional SEO is still needed to build foundational organic search engine visibility, while paid campaigns can generate immediate awareness. AEO ensures that when buyers ask AI engines for recommendations, your brand is in the conversation. Put another way: SEO helps you get found, AEO helps you get chosen.

Why AEO matters now

AI search results are increasingly zero‑click: more than 60 percent of queries end without a user visiting a website. Voice and conversational queries are growing. Consumers expect quick, authoritative answers and don’t have patience to sift through ten links. Without AEO, even strong SEO performance may not translate into visibility within AI‑generated answers. Companies that invest early will be first to gain brand mentions and build authority in this new channel.

What an AEO agency does

Research for AEO

A professional AEO agency helps brands build and maintain answer‑readiness across content, entities and technology. While specific offerings vary, most agency service menus include three pillars: content and knowledge‑base alignment, structured data and entity optimization, and measurement and iteration.

Content + knowledge base alignment for AI answers

 

    • Identify question and topic clusters. An AEO agency researches the exact questions potential customers ask in AI engines. They map these to intent patterns and buyer stages.

 

    • Optimize content structure. Agencies rework existing pages and create new content so that AI models can easily extract answers. This includes adding clear headings, summaries, FAQs, and call‑outs aligned with question clusters.

 

    • Refresh and consolidate content. They decide which pages to update, consolidate or retire to strengthen entity clarity and avoid conflicting information.

 

    • Align knowledge bases and documentation. For SaaS and technical companies, agencies ensure product docs, help centers and blog posts use consistent terminology and entity relationships. This alignment increases the chance that AI engines cite your official content rather than random third‑party references.

Structured data, entities and citations/mentions

aeo in los angeles

 

    • Entity graph development. The agency builds a clear entity graph that defines your company, products, modules and related concepts. This helps AI models understand how your offerings relate to one another and to industry categories.

 

    • Semantic optimization. Agencies go beyond keyword tweaks and optimize how meaning is expressed on the page so AI systems can encode your content into high-quality vector embeddings. Ensuring your pages are more likely to land “closer” to relevant queries in a vector index, improving semantic retrieval and increasing the chances your brand is surfaced in AI-driven search results and answers.

 

    • Schema implementation. They implement structured data (FAQ, Product, Organization, HowTo, Review schema) that follows Google’s structured data guidelines and ensures AI engines interpret your pages correctly. Technical audits fix schema gaps and duplicate information.

 

    • Citation and authority building. Agencies pursue high‑quality backlinks, guest posts, and digital PR to increase your domain authority and provide external signals for AI engines. They monitor brand mentions across AI platforms to ensure citations reflect accurate and positive information.

Measurement and iteration

AEO illustration

 

    • AI visibility tracking. Traditional SEO metrics aren’t enough for AEO. Good agencies use specialized tools to track how often your brand is cited across AI Overviews, ChatGPT Search, Perplexity and Copilot. Metrics include citation frequency, position prominence, and context accuracy.

 

    • Quality indicators. They analyze whether AI mentions capture the right details (use cases, differentiators) and whether sentiment is positive or neutral.

 

    • Impact measurement. Agencies correlate AI visibility with pipeline growth—trials, demos, sign‑ups and revenue. They use UTM‑tagged links and branded search lift to see whether AI references drive conversions.

 

    • Continuous improvement. As AI models evolve, the agency updates your schema, entities and content. They monitor AI outputs for misinformation and correct outdated product references. This ongoing cycle ensures your AEO program stays effective.

What type of clients do AEO agencies specialize in

Digital search

AEO is most impactful for businesses whose customers rely on research and comparisons before purchasing. Common client types include:

 

    • B2B SaaS and technology companies. Buyers often ask “best X software” or “alternatives to Y,” so being present in AI comparisons directly influences demos and trials.

 

    • Professional services and agencies. Firms offering marketing, design or consulting services must appear in voice and conversational queries; AEO helps them become part of recommendation lists.

 

    • High‑consideration consumer products. Medical devices, financial products and complex home goods benefit when AI engines highlight their features and differentiators in short summaries.

 

    • Local service providers. For restaurants, clinics, law firms and tradespeople, AI answers influence local “near me” queries. Over time, voice and AI searches will surface top businesses for these categories.

Does an AEO agency work with startups? What to expect

Yes. Startups and early‑stage companies can leverage AEO to gain visibility without large advertising budgets. According to startup guides, AEO allows young companies to become recommendable rather than merely searchable. Expect the agency to:

 

    • Prioritize the most impactful products or ICP segments first.

    • Build entity definitions and schema from scratch, aligning them with your go‑to‑market positioning.

    • Educate your team about maintaining consistent product descriptions across marketing, product and customer success.

    • Provide a realistic timeline: early citations and entity clarity may appear within 60–90 days, while category dominance and AI comparisons often take several months.

FAQs

FAQ

Does AEO replace SEO? No. AEO/GEO complements SEO. SEO still drives organic traffic and covers content formats that AI answers may not surface. AEO focuses on making sure AI engines correctly understand and cite your content.

How long does it take to see results? Early improvements in entity clarity and schema validation can appear within 60–90 days. Placement in AI comparisons and category queries may take multiple refresh cycles as AI models update their knowledge.

Is there a fixed budget? Budgets for AEO resemble specialized technical SEO retainers. Costs vary based on complexity and number of product lines. A strong agency will scope based on technical depth rather than simple page counts.

Do I need internal technical staff? You’ll need light involvement from product and content leaders for approvals and subject‑matter expertise. AEO agencies handle strategy and execution.

What happens if AI engines misinterpret my brand? An AEO agency will monitor AI outputs and correct misinformation by updating schema, entities and content.

How to hire an AEO agency for marketing services

To hire an AEO agency for marketing services, evaluate how well each partner understands AI search behaviour, entity modelling and multi‑engine visibility before you sign a contract. Below are key questions and proof requests to guide your hiring process.

What to ask during discovery

Use these questions to assess whether the agency truly specializes in AEO or just repackages SEO services:

 

    1. What outcomes will AEO deliver in the first 6–12 months? Ask how the agency will influence visibility in AI comparisons and how they’ll tie that to trials, demos and revenue.

    1. Which ICP segments and products will you prioritize? Strong partners focus on the ICPs and use cases that drive the most value.

    1. How will you map our ICP’s intent patterns and buyer journeys? Look for agencies that analyze AI queries and align content, schema and entities accordingly.

    1. How will you align AEO with existing SEO, paid media and lifecycle marketing? The agency should unify teams around shared entities and intent clusters.

    1. What makes you the best AEO agency for our industry? Ask for evidence of industry‑specific success, methodology transparency and clear expectations.

    1. How will you structure our entities and schema? Evaluate their process for building entity graphs and implementing structured data.

    1. How will you position us in AI‑generated comparisons? They should map competitors, categories and differentiators, then optimize content for extraction.

    1. What technical improvements will you prioritize? A good partner fixes schema gaps, improves internal linking and addresses duplicate infor

    1. How will you create answer‑ready content? Look for structured Q&A formats, concise summaries and entity‑aligned messaging.

    1. How will you monitor and correct AI misinformation? The agency must proactively check AI outputs and update content accordingly.

    1. How will you decide what to update, consolidate or retire? Strong partners audit existing pages, consolidate overlapping content and retire outdated material.

    1. How will you adapt as models evolve? Continuous monitoring and iteration are crucial.

    1. What data and benchmarks do you use? They should track citations, category placement and competitor visibility.

    1. How will you handle multi‑region or international AEO? Ask about region‑specific intent clusters and localized schema.

    1. How will you evaluate competitors? Insightful partners track competitor visibility and identify opportunities.

    1. How will SMEs contribute? They should interview your product and sales experts to capture nuanced features.

    1. How will you manage cross‑functional collaboration? Look for streamlined workflows that minimize your team’s workload.

    1. What does your 90‑day roadmap look like? A clear plan should prioritize schema fixes, content updates and monitoring.

    1. How will you report on progress before revenue impact? Early metrics include citation growth, entity accuracy and AI category presence.

    1. What level of access and communication can we expect? Monthly or biweekly check‑ins and transparent dashboards are standard.

    1. What ongoing maintenance will be required? AEO isn’t a one‑time project, continuous updates keep your brand in AI engines’ knowledge graphs.

What proof to request

Researchers analyzing answers for AEO

To verify claims, request evidence that demonstrates the agency’s AEO capabilities:

 

    • Visibility dashboards. Ask for anonymized dashboards showing citation frequency, summary prominence and category placement across multiple AI engines.

 

    • Entity graphs and schema validation reports. These should illustrate how the agency defined a client’s entities and implemented structured data, along with resulting improvements in AI outputs.

 

    • Client testimonials and references. Seek testimonials that mention measurable outcomes, such as increases in demos or trials triggered by AI visibility.

 

    • Methodology documentation. Reputable agencies provide outlines of their workflow (intent mapping, schema implementation, content reformatting, monitoring, reporting) so you know exactly what to expect.

AEO Action Checklist

Use this quick checklist to start your own AEO program or evaluate a potential partner:

 

    • Audit your existing content and identify question‑based queries relevant to your audience.

    • Define your company, product and concept entities clearly across all pages and profiles.

    • Implement FAQ and Product schema using Google’s structured data guidelines.

    • Refresh and structure pages with clear summaries, question‑and‑answer sections and consistent terminology.

    • Monitor AI visibility and adjust content and schema regularly as models and queries evolve.

Conclusion and next steps

GEO Optimization

AI‑driven search is no longer a novelty, it’s quickly becoming the default way people get answers, compare products and make purchasing decisions. Traditional SEO remains important, but alone it cannot guarantee your brand will appear in AI‑generated answers. Answer Engine Optimization, also known as Generative Engine Optimization, bridges that gap by ensuring your content, entities and authority are recognized and cited by AI systems.

Whether you’re a B2B SaaS provider, a professional services firm or a local business, investing in AEO will ensure you’re not left behind as AI transforms the search landscape.

If you’re ready to see how AEO can accelerate your growth, book an AEO discovery call with SEO Rank Media. During this session a specialist will evaluate your current AI visibility, identify gaps in entity definitions and structured data, and show you what it will take to put your brand at the center of AI‑generated answers. The sooner you start, the sooner you’ll be chosen by answer engines.

ChatGPT Shopping Is Here- And Why It Matters

Caleb Turner on November 3, 2025

OpenAI’s Agentic Commerce approach lets ChatGPT understand your product catalog, present your items in shopping conversations, and, if you’re an approved merchant, complete purchases with Instant Checkout. You participate by providing a structured Product Feed and, if you opt in to purchases, lightweight checkout endpoints and a payment path. The feed is the assistant’s structured source of truth for your products; the spec accepts CSV, TSV, XML, or JSON and allows updates as often as every fifteen minutes so pricing and inventory stay fresh. For brands working with agencies like SEO Rank Media, this means keeping a clean, regularly refreshed catalog export and wiring up the minimal checkout endpoints when you’re ready.

Where this diverges from the traditional search mindset is intent and format. The goal isn’t just to match a query; it’s to help the assistant reason about fit, trade-offs, and next best products in a conversation. That’s why the feed includes fields for relationships (accessories, bundles), review content, and media like video and 3D models. It’s also why there are per-product control flags, so you can allow discovery without enabling checkout, or enable checkout only where you’re operationally confident.

Momentum matters here, too. OpenAI has publicly described merchant onboarding in the United States, Etsy availability inside ChatGPT, Shopify support coming, and a headline partnership with a major national retailer to enable shopping via ChatGPT. In other words, this is not a lab demo; it’s becoming a mainstream retail surface.

What’s actually different from a Google Shopping feed

Google search results page for the query bullet journal.

Google Merchant Center’s base product feed is mature and widely adopted. It supports identifiers, variant grouping, sale windows, and even 3D model links for certain categories. It also enforces parity between the feed and your landing pages; Google crawls and compares price and availability, and it can disapprove products for mismatches.

OpenAI’s feed is built for a conversational assistant. It encourages you to author narrative descriptions, include related products and compatibility, and attach reviews and Q&A inline so the model can quote real customer language to address objections. Regional price and availability can live right in the same product record. And visibility and checkout are toggled per SKU via control flags, giving you more granular participation.

Comparison chart: what’s in the base feed vs. what requires extras

bar chart comparing the features of the OpenAI Product Feed and the Google Merchant Center

Legend: 1 = in the base product feed; 0.5 = supported via a separate feed or API; 0 = not supported

(The chart shows both feeds include core identifiers, variants, sales windows, and preorders. Google handles product ratings and regional inventory through separate mechanisms, so those are shown as partial support. OpenAI’s base feed includes reviews and Q&A inline, explicit related-product relationships, geo-overrides within the same record, and per-SKU participation flags for search and checkout.)

Capability / AttributeOpenAI Product Feed (base)Google Merchant Center (base)Google Separate Mechanism (if any)Notes
Core identifiers (GTIN, MPN, brand)YesYes—Both ecosystems rely on standard identifiers for matching and deduping.
Variant grouping (parent/child variants)YesYes—Use a stable parent grouping to present color/size options as one product.
Sale price + effective date windowYesYes—Always pair sale price with its active start/end window.
Preorder / backorder + availability dateYes (preorder date required)Yes (preorder/backorder with date)—Both require a future availability_date when you mark preorder; GMC also supports backorder.
Inventory quantityYes (inline)NoYes (Content/Merchant API)GMC uses inventory endpoints (incl. regionalinventory) rather than a quantity in the base feed.
Regional price and availabilityYes (inline geo overrides)No (in base)Yes (regionalinventory via Content/Merchant API)Google manages regional overrides through its API and regions service.
Reviews & ratings in product feedYes (inline review stats / Q&A)No (in base)Yes (separate Product Ratings feed)GMC ingests ratings via a distinct reviews feed; OpenAI accepts review data in the product feed.
Related products & relationships (accessories, substitutes, “often_bought_with”)Yes (inline relationships)No—OpenAI exposes explicit relationship fields to power conversational cross-sell.
3D model linkYes (model_3d_link)Yes (virtual_model_link)—GMC’s 3D attribute is currently scoped; OpenAI lists 3D as a media field in the base feed.
Video link in base feedYes (video_link)Limited (not a standard base attribute)—OpenAI’s base spec includes video URLs; Google handles rich media through other paths, not the simple base feed.
Per-SKU discovery & checkout flagsYes (enable_search,enable_checkout)No—OpenAI provides channel-specific participation controls per product.
Site-parity enforcement (price/availability)Not described as a crawl-enforced requirementYes (site ↔ feed parity required)—GMC explicitly checks for price/availability mismatches and can disapprove items.

Why SEOs and growth teams should treat this as a separate feed

Google Sheets spreadsheet titled chatgpt_product_feed_with_narrative

First, narrative and semantics matter. The assistant isn’t just matching strings—it’s explaining. Well-written product descriptions, explicit benefits, trade-offs, and voice-of-customer snippets give it better material to work with. You can also declare relationships such as “required_part,” “often_bought_with,” or “compatible_with,” which helps the assistant build a smarter basket in one conversation.

Second, unification reduces friction. Instead of juggling separate files for product, reviews, and regional overrides, you can put complete context in one feed. That simplifies your pipeline and ensures the assistant always has the full picture.

Third, freshness is a visibility lever. If you update price or stock, you can push those changes quickly so the assistant never promotes items that are out of stock or mispriced. Frequent updates are supported and expected.

Fourth, there are real channel differences. Google requires strict parity between feed and website. OpenAI’s guidance, by contrast, centers the feed as the structured source of truth for conversational use. Treat them as distinct channels with distinct goals and editorial constraints, not a single file you copy and paste.

What this looks like operationally (a practical loop)

A typical team sets up a fifteen-minute job that builds a single authoritative export. The job joins product data with inventory, pricing, regional overrides, media, relationships, and review snippets or aggregates. The exporter validates key rules: currency codes, sale windows, working image and video links, and preorder dates when applicable. It then serializes to CSV or JSON and posts the file over encrypted HTTPS to the allow-listed endpoint. On success, it archives the file and logs the record count and checksum; on failure, it retries with backoff and raises an alert. Many teams begin with full refreshes and later shift to delta exports once volume grows.

To phase in purchases, keep discovery enabled while checkout is off, then enable checkout per SKU once you’re confident in stock accuracy, shipping options, and tax logic. Payments are delegated to your processor; the assistant renders the UI, but you authorize, capture, settle, and refund on your rails. If this sounds complicated, reach out to us and we’ll help you.

Field-by-field: what each attribute means (plain English)

openai feed product list
  • id: Your stable internal SKU or product identifier. Do not recycle it; this anchors updates and prevents duplicates.
  • title: The product name customers expect to see. Keep it specific and readable.
  • description: A clear, scannable explanation of what the product is, what it does, who it’s for, and why it’s different. Treat this like conversion copy, not a keyword dump.
  • brand: The brand or manufacturer. Consistency here improves matching and trust.
  • gtin: The global trade item number, such as a UPC or EAN. When available, this is the most reliable cross-system identifier for exact product matches and review association.
  • mpn: The manufacturer part number. Use it with brand if you lack a GTIN.
  • image_link and additional_image_link: The primary product image and any extra angles or lifestyle shots. Assets should be accurate for the variant (color, size) and high quality.
  • video_link: A product video for demonstrations, comparisons, or unboxing. Helpful for the assistant to surface richer explanations.
  • model_3d_link or virtual_model_link (terminology varies by platform): A 3D asset the assistant or storefront can reference for interactive views or augmented reality in supported categories.
  • price: The regular price with currency code.
  • sale_price and sale_price_effective_date: A promotional price and the start/end window when it applies. Always pair these so the assistant knows when the discount is valid.
  • availability: Stock status such as in_stock, out_of_stock, preorder, or backorder (terminology varies by platform).
  • availability_date: When a preorder or backorder is expected to ship. Required when you mark something as preorder.
  • inventory_quantity: Your current stock count. In OpenAI’s feed this helps the assistant avoid recommending something you can’t fulfill.
  • condition: Usually “new,” but other values exist for refurbished and used goods where allowed.
  • item_group_id: The parent code that ties variants together. Use the same value across color and size variants so the assistant can show a single product with selectable options.
  • color / size / size_system / gender: Variant attributes, especially important in apparel and footwear. Keep naming consistent across the group.
  • shipping: Your shipping methods, regions, and prices in a structured format. Use it to expose free-shipping thresholds and expedited options where they help conversion.
  • delivery_estimate: When the customer can expect the item to arrive. The assistant can use this to answer timing questions.
  • geo_price and geo_availability: Region-specific overrides for price and stock. Handy for cross-border or multi-state operations with local pricing or inventory pools.
  • seller_name / seller_url / policy fields: Merchandising and trust elements—your store name, privacy policy, terms, and return policy. If you enable checkout, many of these become required.
  • product_review_count / product_review_rating: A quick summary of review volume and score to aid ranking and give the assistant context.
  • raw_review_data or q_and_a: Snippets of customer reviews or curated questions and answers. This is powerful: the assistant can quote or paraphrase real buyer language to resolve objections.
  • related_products and relationship_type: Cross-sell and compatibility. Examples include required accessories, compatible parts, alternatives, or bundles. This is how you encode the buying graph you want the assistant to use.
  • enable_search: Whether a product is eligible to surface in ChatGPT shopping results.
  • enable_checkout: Whether the product is eligible for in-chat purchasing. These two flags let you stage participation without committing everything at once.
  • created_at / updated_at: Timestamps for auditing and freshness logic. Keep them accurate to make incremental updates reliable.

    A short, practical checklist

  • Maintain a separate OpenAI feed so you can write conversational descriptions, include reviews and Q&A inline, define relationships, and set participation flags per SKU.
  • Keep identifiers clean: GTIN where available; otherwise brand plus MPN; never recycle id.
  • Pair sale_price with sale_price_effective_date; include availability_date for preorders; validate currency codes.
  • Update whenever product, price, or availability changes—fifteen-minute exports are supported and recommended.
  • Start with discovery enabled and checkout off; enable checkout per SKU once stock, taxes, and shipping are proven.

Closing thought

Previously, your shopping feed was a supporting document that needed to line up with your PDPs and help an ad platform target the right queries. In the agentic era, your feed is the main character: it’s the content the assistant reads, reasons over, and uses to recommend the right bundle, answer the buyer’s objections, and—in many cases—complete the order. That’s why this isn’t “just SEO.” It’s the connective tissue between product truth, persuasive storytelling, and a new, conversational shopping surface.

The llms.txt debate: what’s real, what’s hype, and how to use it (safely)

Caleb Turner on October 9, 2025

Short version:
llms.txt is a proposal, not a formal web standard. Some sites ship it; some AI tools and SEO plugins promote it. Google says you don’t need it to show up in AI Overviews or any other Google products.

But: …like any public .txt file, an llms.txt URL can be crawled and indexed, so in rare cases a bare, unstyled text page could show up in search instead of your real, designed page.

I know of someone who created a playful file called cats.txt to make a simple point: Google can index plain text files if they’re publicly accessible and discoverable. In other words, the name doesn’t matter, if it’s a .txt and reachable, it can show up in search, just like any other indexable file type.

If a text file is publicly visible on your site, search engines can list it, which you certainly don’t want. To prevent this, send X-Robots-Tag: noindex in the HTTP response (works for non-HTML files), and if you want to point search engines to the right page, add a Link: <https://your-page>; rel=”canonical” header. Here’s what a real plain-text doc looks like in a browser (the kind that could hypothetically rank if you don’t block indexing):
 https://developers.cloudflare.com/llms.txt.

Google’s documentation covers both the noindex response header for non-HTML resources and using an HTTP canonical header.

Below is a clear walk-through, first in plain English (with analogies), then the technical details and safest practices.

What is llms.txt?

llms.txt is a simple text file you put at example.com/llms.txt. The file lists or summarizes your most important pages in Markdown, so AI systems can more easily read and use your content. Think of it like a cheat-sheet menu for bots: “Here are the dishes you should try; here’s how to ingest or interpret them.” It’s inspired by robots.txt and sitemaps, but it isn’t an official protocol like those are, just a community proposal.

Today, there’s no universal adoption. A directory of live implementations shows many developer-tooling/docs sites experimenting, often paired with a bigger llms-full.txt that expands the content.

John Mueller / Google’s stance:  In a conversation with Caleb and a few other colleagues, Google’s John Mueller made it clear what side of the debate he’s on. You don’t need llms.txt; the guidance is to keep following normal SEO practices.

Why is there a debate?

John Mueller comment highlighting the llms.txt debate around SEO tools, AI search hype, and whether websites need the file.

The “we don’t need this” camp

  • Google says it won’t use llms.txt for AI search results, so for many sites this is optional at best.
  • llms.txt isn’t a standard; support and behavior differ by bot. In other words, don’t expect consistent results.

The “we’re seeing activity in the wild” camp

  • Practitioners have shared examples and logs showing Google indexing llms.txt pages, alongside surges of bot hits when large platforms roll it out, because it’s just a public text file like any other.
  • That doesn’t prove Google uses llms.txt for AI search results. It only proves that public .txt files can be crawled and indexed, which Google’s docs have said for years.

A layperson’s guide to the technical bits

  • robots.txt vs noindex
    robots.txt     is like a bouncer who tells certain crawlers not to walk certain halls. It doesn’t guarantee your URLs won’t show up in the search phone book (the index). Pages blocked in robots can still be indexed by URL if they’re linked elsewhere. If you truly don’t want a URL in the index, use noindex, that’s a separate rule delivered in HTML or HTTP headers.
  • X-Robots-Tag and canonical
    For non-HTML files like .txt, the right place to control indexing is the HTTP response header.

    • X-Robots-Tag: noindex = “don’t list this address in Google.”
    • Link: <https://example.com/page>; rel=”canonical” = “if you do need to reference this, use our public address here.” (Google supports canonical in the header for non-HTML formats.)

  • User-Agent/UA and why blocking by UA isn’t enough
    “UA” is the name badge a crawler shows at the door (e.g., Googlebot, GPTBot). You can write per-UA rules in robots.txt, and major AI vendors document their UA strings.

But name badges can be forged; to be sure a request is truly a vendor’s bot, verify by reverse-DNS/IP, not just the UA string. Cloudflare has even accused some AI crawlers of stealth crawling (changing badges and IPs).

  • “Cloaking” and dynamic rendering
    Serving different content to bots than to users is a slippery slope. Google considers cloaking a spam tactic when bots and users see materially different things. Google also deprecated dynamic rendering (bot-only HTML) as a long-term approach. If you want a bot-friendly version, keep the substance the same as what people see.

The cannibalization risk (and how to avoid it)

User reviewing a raw indexed page showing the cannibalization risk of llms.txt replacing a designed website page in search results.

Imagine your llms.txt ranks for a branded query. A searcher clicks and lands on a wall of plain text with no design, navigation or conversion paths. That’s the risk: poor UX and lost revenue. It’s not hypothetical, plain text files do get indexed, and practitioners have shown real examples of llms.txt and llms-full.txt pages in the index. We also showed an example earlier in this article.

Fix: keep llms.txt fetchable (so AI tools can read it) but non-indexable in web search with:

X-Robots-Tag: noindex

Link: <https://www.example.com/your-preferred-page>; rel=”canonical”

These are HTTP headers on the llms.txt response, not tags inside the file. This is the safest, standards-compliant way to prevent cannibalization while still letting crawlers pull your file.

If you still want to experiment: safest practices

SEO analyst reviewing safe llms.txt testing practices with analytics before experimenting with AI search visibility.

1) Treat llms.txt as optional, and experimental

Ship it only if it supports real goals. Keep expectations modest; it’s a proposal, not a protocol.

2) Prevent web-search cannibalization

Serve HTTP headers on llms.txt (and llms-full.txt if you publish one):

  • X-Robots-Tag: noindex
  • Optional: Link: <https://www.example.com/the-main-URL>; rel=”canonical”

Think: “Please don’t list this file in the phone book; if you must reference something, here’s the main storefront.”

3) If you want to block certain AI crawlers elsewhere, do it the right way

  • In robots.txt, write rules per UA (e.g., User-agent: GPTBot). Vendors like OpenAI document their bot names.
  • For high-stakes data, verify IPs (reverse-DNS) because UA strings can be faked. Google documents how to verify Googlebot; similar logic applies to others.
  • Be aware: some AI bots have been accused of ignoring robots.txt or crawling stealthily, so consider edge-level blocking if needed (WAF/CDN).

4) What to put in llms.txt (if you use it)

Link to canonical, public pages that you want AI systems to cite: FAQs, policies, product specs, pricing explainer, and key how-tos. Keep it concise; don’t dump the whole site.

5) Instrumentation & monitoring

  • A .txt file is just raw text. Web browsers don’t run code inside it, so you can’t drop a JavaScript analytics snippet (like GA/GTAG) into a .txt and expect it to fire. Browsers only execute scripts when the content is served as a script/HTML type, not text/plain.

If you still want to see who’s fetching that file, look at your server or CDN access logs. Those logs list every request (time, IP, user-agent, URL, etc.), so you can count hits to /llms.txt even without JavaScript. Examples: Apache’s access log and Cloudflare Logs.

  • Watch Search Console: if a text file starts appearing in “Indexed,” revisit your headers. Google’s docs confirm indexing can occur even without crawling the content (e.g., when discovered by links).

6) Don’t block JS/CSS for Googlebot

If your SEO defense plan includes blocking scripts to hide unique content from AI, be careful: blocking JS/CSS broadly can break rendering in Google Search. If you must, target AI bots individually, not Googlebot

The bottom line (for decision-makers)

Business decision-maker weighing the pros and cons of llms.txt for AI search strategy and safe SEO implementation.
  • Not required: Google’s AI Mode doesn’t depend on llms.txt; normal SEO still wins.
  • Not a standard: It’s a proposal with uneven support. Useful for experimenting, especially for docs-heavy products; not a silver bullet.
  • If you try it, ship it safely:

    • Put it at the root.

      Keep it short and link to your best pages.

      Send X-Robots-Tag: noindex and, if helpful, a header canonical.

      Keep content parity; avoid UA-based “special versions” that diverge.
    • If you must block certain AI bots elsewhere, use per-UA robots rules plus IP verification at the edge; be aware of stealth crawlers.

If you want a place to start, you can base your evaluation on current adoption (developer docs ecosystems, directories of live files) and any internal log evidence you have about bot hits to llms.txt. Then decide whether it’s worth maintaining a curated cheat-sheet for AI, or whether your time’s better spent doubling down on structure, internal links, and copy, the proven levers. SEO Rank Media is among the leaders in the AI search conversation. Reach out to explore how we can set your brand up for the future.

The AI Era: Why Search Engines Aren’t Going Anywhere

Caleb Turner on September 22, 2025

There’s a common misunderstanding that large language models (LLMs) like ChatGPT or Gemini are replacing search engines. They aren’t. LLMs change how results are presented and explained, but the heavy lifting of finding, organizing, and ranking the web still belongs to search engines. In plain English: LLMs are the brainy librarians inside of a giant library; search engines are the library’s cataloging system that keeps track of every book, page, and shelf.

Below is a clear look at what each does, why they’re different, and why search is not only sticking around but also growing.

What search engines actually do (and why that matters)

Search engines run a huge, ongoing pipeline that works like this:

  1. Crawl: Automated bots (“crawlers”) visit web pages and take notes on what they find.
  2. Index: Those notes are stored in a gigantic, constantly updated catalog (the “index”).
  3. Rank & Serve: When you search, the engine looks up the most relevant pages in that index and ranks them using complex algorithms.

Google’s own documentation lays out this crawl → index → rank process in detail. If you’ve never read it, it’s surprisingly readable and shows the scope and complexity behind what looks like a simple search box. directly;

You can’t browse Google’s index directly, it’s proprietary and unimaginably large. You query it. If you own a website, you can see your slice of the index in Google Search Console’s Page indexing report, which shows which of your pages are in or out and why. Microsoft offers similar visibility in Bing Webmaster Tools, including a Sitemap Index Coverage report that flags reasons URLs are excluded.

This is the invisible machinery of the open web. It’s what makes it possible to find new content minutes after it’s published and to keep billions of pages ordered enough to be useful.

What LLMs actually do (and what they don’t)

LLMs are trained to predict and compose text. They’re excellent at summarizing, explaining, reformatting, and reasoning over information they’re given. But there are two common misunderstandings:

  • LLMs do not maintain a live, internet-wide search index. The model itself isn’t crawling the web in real time or keeping a searchable catalog of every page like a search engine does. When LLMs need fresh facts, they typically consult a search engine index. Meaning they call a search engine service (UI or API). The search engine then queries its own index, returns ranked results, and the LLM fetches a few of those pages and combines them into the answer it generates for the user. Google literally calls this “grounding with Google Search.”
  • “Browsing” ≠ “crawling.” What we just described is called retrieval and summarization, not operating a global crawler and index. OpenAI’s newer “deep research” mode, for example, plans multi-step lookups and shows sources. Again: retrieval plus synthesis, not running its own universal web index.

This distinction matters because it explains why LLM answers can be hallucinatory. Without a high-quality retrieval step (i.e., search), an LLM is just “guessing” based on training data that could be outdated or incomplete.

That said, ChatGPT (the product of OpenAI) now runs a real web crawler called OAI-SearchBot and maintains OpenAI’s own web index so it can discover pages and show them as cited sources in ChatGPT Search. Which again proves this article’s point: you still need search infrastructure under the LLM.

The winning combo: grounding LLMs with search

The industry term for blending search with generation is Retrieval-Augmented Generation (RAG). In RAG, the system first retrieves relevant documents from a trusted source (like a search index or an enterprise knowledge base) and then generates an answer that cites those sources. Requiring the AI search engine to cite its sources can also dramatically reduce hallucinations. The original RAG research popularized this approach in 2020, and it’s now widely used.

You’ll see this philosophy in multiple places:

  • Google Gemini / AI Overviews: “Grounding with Google Search” pipes real-time search results into the model and returns answers with citations.
  • Vertex AI: Google Cloud’s guidance explicitly recommends grounding model outputs in verifiable data, via Search, RAG, Maps, and more, to reduce hallucinations.

The big picture: LLMs are the presentation and reasoning layer; search is the fact-finding and verification layer. You need both.

The library and the librarian

Think of the web as a giant library:

  • The search engine builds and maintains the card catalog (the index). It constantly scans new “books” (web pages), decides where they belong, and keeps the catalog current.
  • The LLM is the librarian who reads the relevant pages you point to and then explains them in friendly language, weaving them into a clear, direct answer. If the librarian is allowed to cite the exact books and page numbers, you can check the work.

When the librarian doesn’t check the catalog first and just “remembers” what books might say, mistakes happen. That’s why modern AI features emphasize grounding and citations.

“But aren’t people just using AI instead of Google now?”

Short answer: no. AI usage is up, and Google Search remains massive and growing.

  • Alphabet’s earnings releases and CEO remarks throughout 2025 show double-digit growth in Search revenue and healthy overall query growth, including a 70% year-over-year jump in Google Lens searches, much of which is incremental (i.e., additional to traditional text queries). That’s expansion, not replacement.
  • Independent financial reporting backs this up: multiple quarters in 2025 attribute Alphabet’s outperformance partly to strength in core search, even as AI features roll out alongside it.

It’s also useful to separate revenue from queries. Revenue grows when users stay engaged and ads remain effective; queries grow when people search more, in more ways. Google has repeatedly highlighted growth in newer, multimodal behavior, like searching with your camera (Lens) or combined gestures, showing search is evolving rather than shrinking.

Why LLMs don’t (and shouldn’t try to) be search engines

  1. Freshness at web scale: The public web adds and changes billions of pages. Keeping a comprehensive, deduplicated, spam-resistant, and continuously updated index is a specialized, infrastructure-heavy job. It’s what search engines were built for.
  2. Transparency and provenance: When an LLM is required to cite sources, users can click and verify. This is standard in grounded systems like Gemini’s “Search grounding” and Vertex’s guidance. Purely generative answers can’t offer the same audit trail.
  3. Governance and site control: Website owners monitor their presence in the index through Google Search Console and Bing Webmaster Tools, diagnosing why pages are in or out. That visibility is essential for a healthy open web and isn’t replaced by a model’s internal training data.
  4. Commercial ecosystems: Search drives measurable, intent-rich traffic that businesses can analyze and optimize. That incentive structure sustains publishing and commerce broadly. The earnings results we’ve seen suggest these dynamics are holding, even as AI features appear in the interface.

What this means for everyday users

  • You’ll see more answers. AI summaries sit on top of search results and often include citations so you can dive deeper. Expect more multimodal options (speak, snap a photo, or draw a circle on your screen) that kick off a search behind the scenes.
  • Quality still wins. If you publish online, the fundamentals matter even more: sitemaps, clean site architecture, crawlability, canonical tags, structured data, and helpful content. Search engines need to index and rank your pages before an LLM can confidently cite them.
  • Trust but verify. AI answers can be great for speed and clarity, but when it counts, click through the citations. Even OpenAI’s more advanced research features emphasize sources precisely because models can still overstate or hallucinate details.

What this means for businesses and publishers

  • Search is still the discovery backbone. Alphabet’s 2025 results show search’s resilience and growth as AI features roll out; the pie is getting bigger, not smaller.
  • Optimize for being cited. When LLMs ground answers, they look for trustworthy, well-structured, crawlable sources. Make sure your pages are indexable and well-labeled so they’re retrieved and cited instead of a forum thread summarizing your work.
  • Expect new query types. Visual and voice-led searches are growing fast, often incrementally—meaning they’re additions to classic typed searches, not replacements. Prepare your content and product data (images, alt text, schema) to be useful in those contexts.

Quick FAQ

Do LLMs “crawl the web”?
No. The applications around LLMs may fetch pages when you ask a question, often via a search partner, but the models themselves don’t operate a global crawler and index like a search engine. Google’s own AI stack explicitly “grounds with Google Search.”

Can I see the web index somewhere?
Not directly. You can query it (e.g., with Google or Bing), and if you own a site, you can inspect your pages’ status in Google Search Console or Bing Webmaster Tools.

Isn’t AI going to reduce searches
Evidence to date suggests the opposite: search usage and revenue are growing while AI features roll out, and newer behaviors like Lens are expanding the pie.

So what’s the right mental model?
Search engines find and rank facts at web scale. LLMs present and reason over those facts. Together, they produce faster, clearer answers, with links you can check.

The bottom line

LLMs have not replaced search; they’ve changed its surface. Underneath any polished AI answer, the classic information-retrieval pipeline, crawling, indexing, retrieval, and ranking, is still doing the heavy lifting. Modern systems combine them: search grounds the answer; the LLM explains it. And if you look at 2025’s numbers and usage patterns, search isn’t going anywhere. It’s evolving, growing, and quietly powering the AI experiences we’re all watching unfold before our eyes. Reach out to SEO Rank Media if you want a partner who understands the direction search is headed and how to position your business to be at the forefront of the evolution.

AI API Test: See What AI Agents Read, Cite, and Trust

Caleb Turner on September 12, 2025

An AI API test shows exactly how AI search engines and assistants (like ChatGPT, Google’s AI Mode, and Perplexity) consume and reuse your content by giving them a clean, structured endpoint and logging what they ask for. Brands that run these tests learn how to feed AIs the right facts, earn more citations and brand mentions in answers, and protect revenue as clicks shift from traditional organic search to AI summaries (Pew finds link-clicks drop sharply when AI summaries appear, while Semrush reports AI searchers convert far better than classic search users).

Key takeaways

  • AI summaries reduce clicks to websites; visibility now means “being the answer,” not just ranking.
  • A simple “honeypot” API reveals what each agent asks, which sources it trusts, and how often it re-checks your data.
  • A dual-feed strategy wins: persuasive HTML for humans, structured JSON for AI agents.
  • Cloudflare Radar now tracks crawl-to-refer ratios by AI bot, evidence that many models consume more than they send back.
  • Semrush’s AI Visibility Index shows which sources each model leans on; community sites (e.g., Reddit) often dominate ChatGPT while Google’s AI Mode favors structured authorities. Optimize for both.

Detailed guide

What is an “AI API test” in plain English?

It’s a small experiment where you (1) detect likely AI agents, (2) offer them a machine-friendly API endpoint with verified facts about your products/services, and (3) log exactly which agents called it, what they asked, and how they used it. This gives you measurement, attribution, and a blueprint for fixing gaps in how AIs talk about your brand. In a field test summarized by Agent Berlin, teams used a honeypot endpoint and could see “fan-out” sub-queries from AI agents, proving what details matter most to machines.

Why is this suddenly business-critical?

Two shifts collided in 2025:

  1. Google’s AI summaries change click behavior—Pew shows users are less likely to click links when AI summaries appear, and they almost never click the cited sources (≈1% of visits). That is a direct threat to traffic.
  2. AI searchers who do engage are high-intent. Recent data reveals LLM search as a conversion engine and urges brands to “be the answer,” not just the blue link.

Translation: you can’t afford to guess what AIs read from your site. You need instrumentation.

What do recent large-scale studies say about AI visibility?

  • Pew Research (2025): ~18% of studied Google searches showed an AI summary; when summaries appeared, users clicked less and often ended the session sooner. Wikipedia/Reddit/YouTube dominate as cited sources.
  • Semrush AI Visibility Index (2025): Tracks brand mentions and cited sources across industries for ChatGPT vs. Google AI Mode and shows they lean on different source ecosystems; the microsite also surfaces “Top 10 Sources” per vertical, helping you prioritize where to earn citations.
  • Cloudflare Radar (2025): Adds AI-bot telemetry (crawl vs. refer) and documents industry worries about “stealth crawling” plus tools to control or charge for AI access. This reinforces why you should measure agent behavior directly with an API test.

How does an AI API test actually work?

How does an AI API test actually work?

At a high level: your server inspects requests (the “User-Agent” header, IP ranges, and other signals). If it looks like an AI agent, you give it short instructions pointing to a special endpoint (the honeypot). That endpoint returns clean JSON facts, product specs, pricing, policies, and store hours—tagged with an agent identifier so you can attribute calls. Tests have observed platform differences: Perplexity often follows API instructions directly, while ChatGPT tends to ask the user for permission first before calling outside APIs.

Is this “cloaking”?

No, if the structured JSON reflects the same facts humans see. You’re not hiding or manipulating content; you’re decluttering it for machines. Keep parity with your human-visible page, and document your intent in your privacy/robots notes.

What should I measure?

  • Call volume per agent (e.g., ChatGPT, Perplexity-User): trend lines reveal who’s reading you.
  • Query themes & “fan-out” sub-questions: Shape your FAQs and product pages around what agents actually ask.
  • Field-level usage frequency: Which JSON fields are read most (price, ingredients, warranty)?
  • Citations/mentions in AI answers: Did the model name your brand and/or cite your URL?
  • Crawl-to-refer ratio: Cloudflare’s new Radar views help you compare model consumption to referral behavior.

What does a minimal setup look like?

  1. Detect agents: check User-Agent and allow-listed IPs where possible. Perplexity documents PerplexityBot and Perplexity-User plus IP JSON; use that as a baseline.
  2. Redirect AIs to a clean endpoint: return a one-liner on the HTML page that politely tells agents to fetch /agents-api?src={agent}.
  3. Serve normalized JSON: consistent field names (e.g., price, in_stock, last_updated).
  4. Log & attribute: require an identifier (query param or header) and record IP/ASN where legal.
  5. Validate truth: cross-check JSON vs. the visible page; consider using time-stamped fields so you can prove freshness if an AI answers with stale data.
  6. Compare answers: ask the same questions on ChatGPT, Perplexity, and Google’s AI Mode and see if they reuse your fields or cite you.

Example / Template (copy & adapt)

Minimal agent endpoint JSON

{

  “brand”: “Acme Widgets”,

  “sku”: “W-42”,

  “price”: 19.99,

  “currency”: “USD”,

  “availability”: “in_stock”,

  “warranty_months”: 24,

  “last_updated”: “2025-09-10T12:00:00Z”,

  “source_url”: “https://www.example.com/widgets/w-42”

}

Attribution tip
Require ?agent= on the endpoint and accept X-Agent as a header fallback, which creates reliable logs even if a crawler spoofs a browser string. The aforementioned honeypot proved this strategy works in practice.

How do I turn test insights into more visibility and revenue?

How do I turn test insights into more visibility and revenue?
  • Give AIs exactly what they seek. If the logs show frequent questions about returns or materials, add those as discrete JSON fields and as scannable bullets on your HTML page. Better answers → more brand mentions and citations in AI summaries.
  • Win the sources AI trusts. Research and determine the top cited sources for your vertical; prioritize partnerships, PR, and content for those domains (e.g., Reddit threads, .gov guidance, and industry reviewers) to increase your odds of inclusion.
  • Treat AI visibility as a funnel. Even if clicks fall on AI summary pages, branded mentions and “as cited by” moments can lift assisted conversions, especially since LLM users tend to be higher-intent. Measure brand search and direct traffic after AI mention spikes.
  • Protect your margins. Cloudflare Radar’s crawl/refer telemetry plus access-control features help you decide who to allow, throttle, or charge, which is useful if training demand outpaces referral value.

How long should the test run, and what’s a “good” sample?

Run at least 2–4 weeks to capture weekday/weekend cycles and product updates. For smaller sites, aim for 100+ attributed agent calls; for enterprise, target 1k+ calls across at least two models. Pair logs with a weekly review of brand mentions in ChatGPT and Google AI Mode.

What about policy and ethics?

Honor robots and published bot IP lists; several providers (Perplexity, OpenAI) document user agents and controls. If you must block training, manage robots and WAF rules thoughtfully; Cloudflare offers features to segment/verify bots and even explore “pay per crawl.” 

Example Box: A dual-feed pattern you can pilot this week

  • Human page: persuasive copy, images, reviews, and rich FAQs.
  • Agent endpoint: normalized JSON of the same facts.
  • Post-deploy checks:
    1. Ask each model a buying-intent question (e.g., “Is W-42 waterproof?”).
    2. Observe whether your JSON field appears verbatim in the answer.
    3. Track if your brand/URL is named or cited.
    4. Adjust fields and repeat next week.

FAQs

Will this help or hurt my SEO?

It helps if the JSON matches what humans see. You’re clarifying, not hiding. AI visibility is its own discipline alongside SEO; treat it as a complementary layer.

What if an AI bot ignores robots?

Measure first (your logs), then decide whether to throttle, challenge, or block. Cloudflare Radar documents concerns and provides controls for AI bots. 

How do I know which sources to court for citations?

SEO Rank Media uses AI visibility tracking tools to see which domains each model most often cites in your niche. Cross-check the lists across tools and models, note the sites that appear repeatedly, and rank them by relevance and authority. We then prioritize outreach and content placements on those high-overlap sources, then monitor citations over time to refine the target list.

Does Perplexity publish its bot details?

Yes. Perplexity lists PerplexityBot and Perplexity-User, plus IP ranges you can allowlist. 

Checklist / TL;DR

  • Detect agents; route them to /agents-api.
  • Serve parity JSON with versioned, time-stamped facts.
  • Log agent, IP/ASN, path, and fields read.
  • Compare AI answers for citations/brand mentions weekly.
  • Target the sources each model favors in your vertical.
  • Monitor crawl-to-refer ratios; adjust allow/block rules.
  • Re-run quarterly as models and policies change.

Ready to win AI answers and local visibility?

Ready to win AI answers and local visibility?

Turn your AI API test insights into traffic, leads, and revenue. Contact SEO Rank Media for AEO (Answer Engine Optimization) and GEO (Generative Engine Optimization) services. We’ll:

  • Audit your site’s AI readiness and structured data
  • Set up & instrument your honeypot/AI API test
  • Build dual-feed content (human HTML + machine JSON)
  • Craft an AEO/GEO roadmap to earn citations in ChatGPT, Google AI Mode, and Perplexity
  • Align your source acquisition plan to what the models actually trust

Get started today with a quick strategy call; SEO Rank Media is ready to help.

How AI Mention Trackers Work: A Clear Guide to Understanding Visibility in Large Language Models

Caleb Turner on July 14, 2025

Artificial intelligence is becoming more deeply embedded in the way users search for, engage with, and consume information online. Businesses are now facing a new visibility frontier: large language models (LLMs) like ChatGPT, Claude, Google’s Gemini, and Perplexity. These AI tools are rapidly shifting how people discover brands and products, but there has long been a missing piece for marketers: how can you measure your brand’s presence across these tools?

Enter AI mention trackers—tools like Profound, Peec AI, and others that are helping brands figure out how often they appear in AI-generated answers. Think of them as the modern-day equivalent of media monitoring tools, but instead of scanning newspapers or websites, they scan what the AIs are “saying” about you. Let’s walk through exactly how these tools work, step by step, in simple and clear terms.

Step 1: Feeding Questions to AI Models

The first thing an AI mention tracker does is simulate real-world user queries. For example, if you sell coffee, it might generate prompts like:

  • “What are the best coffee brands for home brewing?”
  • “Which companies sell sustainable coffee beans?”

These questions are either preloaded by the tool or customized by the user. Then, the tool asks these questions to various AI platforms—ChatGPT, Claude, Perplexity, and others. These queries are sent using APIs or simulated browser sessions, mimicking the behavior of a real user.

To make the results more robust, the tool may vary how it phrases the questions, capturing a wider net of responses. This ensures the data reflects how real users might engage with AI tools.

Step 2: Collecting the AI’s Answers

Once the questions are submitted, the AI models reply with natural-language answers. The tracker collects all of these answers—a big pool of unstructured text. If the AI provides source citations or links (as Bing or Google often do), the tool grabs those too.

This phase is about capturing everything that the AI outputs, regardless of whether your brand appears yet.

Step 3: Detecting Brand Mentions

Now comes the scanning. The tool searches through each AI-generated answer looking for specific brand names, website URLs, or product terms. It checks to see if, for example, “Acme Coffee” or “acmecoffee.com” shows up in the text.

This is similar to a human pressing “Ctrl+F” and looking for their company’s name. The tool notes:

  • Where the mention appeared
  • How often it appeared
  • In what context (Was it a top recommendation? Just a mention in passing?)

If the brand doesn’t appear, that’s recorded too. These “non-mentions” are equally important because they show where the AI isn’t recognizing your brand.

Step 4: Counting and Aggregating Mentions

Counting and Aggregating Mentions

The tracker now tallies up the results across many queries and platforms. This helps quantify your brand’s visibility. You might learn that your brand appeared in:

  • 8 out of 20 questions on ChatGPT
  • 10 out of 20 on Google Gemini
  • Only 3 out of 20 on Bing Chat

These numbers are typically translated into metrics like “share of voice” (SOV) or mention frequency. Tools like Profound display this in an easy-to-read dashboard, comparing your visibility to your competitors.

Over time, this creates trend lines that show whether your brand’s AI visibility is improving or declining.

Step 5: Attributing Mentions to Sources

A crucial part of these tools is identifying why an AI mentioned your brand. In many cases, it’s because of external sources cited by the AI model. For example:

  • Bing Chat might footnote your brand with a link to a popular review site
  • Google’s AI Overviews might mention your company and cite your blog or Wikipedia

The tracking tool records these citations and links them to your mentions. This is called “citation analysis.” It helps you understand which articles, websites, or publications are fueling your AI visibility.

When an AI doesn’t mention you but mentions a competitor, these tools can also highlight what sources were cited for them. This gives you ideas about where you might need more coverage.

Step 6: Presenting the Results

Presenting the Results

All of this data gets organized into a simple dashboard. It might tell you:

  • Your brand was mentioned in 40% of answers about “best coffee brands” on ChatGPT this month
  • That’s up from 30% the month before
  • The most frequently cited source was HomeBarista.com
  • Competitor JavaWorld appeared more often than you on Google SGE

Some tools also analyze sentiment: whether the AI’s tone was positive, neutral, or negative about your brand. While more advanced, this adds another layer to understanding your visibility.

A Real-Life Example: Acme Coffee

Imagine you run a fictional brand called Acme Coffee. You want to know if AI tools are recommending you when people ask about coffee.

  1. The tracker sends prompts like “What are the best coffee brands?” to ChatGPT, Claude, Google Gemini, and Bing Chat.
  2. ChatGPT responds with: “Some great coffee brands are Acme Coffee, BeanCo, and JavaWorld.” The tool flags that Acme was mentioned.
  3. Google’s AI says: “According to HomeBarista.com, Acme Coffee roasts top-tier beans.” The tool notes the mention and attributes the source.
  4. Bing Chat doesn’t mention Acme at all but includes JavaWorld. That’s also important intel.

After querying multiple questions and platforms, the tracker produces a report:

  • Acme was mentioned in 7 out of 10 queries on ChatGPT
  • 5 out of 10 on Google Gemini
  • 3 out of 10 on Bing Chat
  • Most Acme mentions cited HomeBarista.com
  • JavaWorld beat Acme by 2 mentions across the board

Tools Like Ahrefs Add Another Layer

Tools Like Ahrefs Add Another Layer

Some platforms, like Ahrefs, take a slightly different but powerful approach. Rather than running queries in real time, Ahrefs leverages a vast existing database of AI responses and questions. You can type in a brand name or topic like “sneakers,” and instantly see a list of relevant queries and AI answers that reference the topic.

This lets you:

  • Identify competitor gaps (queries where your competitors show up but you don’t)
  • Discover new topic opportunities (queries you never thought of that relate to your niche)

This retrospective approach complements real-time trackers like Profound or Peec AI by giving you a broader strategic view.

Tracking LLM Traffic in GA4: Why It Matters

AI visibility isn’t just theoretical. Brands are already seeing meaningful traffic driven by AI tools. Tracking this traffic in Google Analytics 4 (GA4) is now essential.

While Google Search Console still blends AI Overview and AI Mode traffic with regular search, GA4 gives you tools to segment this data more precisely.

Two Main Tracking Approaches:

  1. GA4 Explore Reports:
    • Create a session segment using a custom regex filter to capture traffic from AI sources like ChatGPT, OpenAI, Copilot, Gemini, Perplexity, etc.
    • Visualize this data with line graphs, bar charts, or tables.
  2. Looker Studio Reports:
    • For detailed reports: Create a new channel group in GA4 for AI traffic.
    • For quicker views: Use the same regex filter in your Looker Studio tables and charts.

These dashboards let you:

  • Track how much traffic is coming from AI tools
  • See which pages are being visited from AI answers
  • Understand whether your AI visibility is translating into real engagement

Final Thoughts: Why This Matters

The future of search is increasingly conversational and AI-driven. Tools like Profound, Peec AI, and Ahrefs help marketers stay ahead by answering this crucial question:

“Are the AIs talking about me?”

If they are, great—you can double down on what’s working. If not, you can take action to increase visibility by improving the content on sites that AIs pull from.

AI mention trackers give marketers, PR pros, and SEOs a crucial lens into how modern algorithms perceive and recommend their brands. By bridging the gap between traditional SEO metrics and AI-powered search behaviors, these tools ensure your strategy remains both measurable and forward-looking.

Start tracking now, and you’ll not only see how often you appear in the AI conversation, you’ll start shaping it.

How do modern AI search engines and LLMs operate and how do you optimize for them?

Caleb Turner on June 2, 2025

This isn’t 2015 anymore, yet some SEO “experts” are still clinging to tactics like they’re waiting for Windows 7 to make a comeback. Modern AI-powered search engines and large language models (LLMs) leverage Retrieval-Augmented Generation (RAG) to combine external data retrieval with text generation, ensuring answers are both current and contextually accurate. By performing a real-time search of trusted documents before crafting a response, these systems mitigate outdated training data and “hallucinations.” To optimize for them, create clear, structured content with up-to-date citations, conversational Q&A headings, and appropriate schema markup, so AI retrieval steps can easily identify and quote your material.

Key Takeaways

      • RAG enables AI to fetch and ground answers in fresh, external sources.

      • Structured Q&A headings and bullet points improve AI snippet retrieval.

      • Embedding authoritative, date-stamped references boosts trust signals.

      • Conversational phrasing and varied keywords aid vector-based matching.

      • Schema markup (FAQPage, HowTo) helps AI isolate self-contained snippets.

      • Off-page promotion can still surface in AI searches.

      • Optimizing content for RAG-driven AI results increases probability to appear in AI summaries and chatbot responses, giving you traffic that static search rankings might miss.

    Detailed Guide

    What is Retrieval-Augmented Generation (RAG) in simple terms?

    retrieval augmented generation

    Retrieval-Augmented Generation (RAG) is a hybrid AI workflow that enhances language models by letting them “look up” relevant documents at query time, rather than relying solely on what they learned during pretraining. Imagine asking a librarian to fetch the latest journal article before answering your question; RAG works similarly. Except this librarian is more like Alexa or Siri than your stereotypical Miss Finster.

    When you submit a query, the system first searches an external data source, such as a website index, a private knowledge base, or a specialized dataset of academic papers, for pertinent passages. Then, it feeds those retrieved snippets into the LLM as additional context, guiding the generative process so the answer is grounded in factual, up-to-date material. This approach addresses two major limitations of standard LLMs: information cutoff dates and the risk of “hallucinations,” where the model invents plausible-sounding but incorrect details.

    How does the retrieval phase work?

        1. User Query Submission
          You ask a question—e.g., “What are the 2025 tax deadlines for small businesses in Texas?” The RAG-enabled system takes this natural-language query as input.

        1. External Search
          Instead of directly generating an answer from pretraining data, the system performs a search against an external document collection, which could be a public web index, a company’s internal file repository, or a specialized dataset of academic papers (AWS, 2024; WEKA, 2025).

        1. Result Ranking
          Retrieved documents or text snippets are ranked by relevance using vector similarity, which transforms both the query and documents into numerical embeddings, or traditional keyword-based matching. The top N results (often broken into smaller “chunks” of text) are selected based on how closely they align with the user’s question.

        1. Outcome
          At the end of this phase, the system holds a set of highly relevant, often date-stamped passages that directly address the query.

      How does the augmentation and generation phase work?

          1. Context Assembly

        The RAG engine takes the top-ranked snippets—sometimes as short as a few sentences each—and concatenates them with the original user query. This assembled context is fed into the LLM.

            1. Guided Response Generation

          Rather than “freewriting” from its pretraining knowledge, the LLM now “reads” the assembled context and composes an answer that weaves together facts from the retrieved snippets with its own linguistic patterns. It essentially uses the retrieved passages as anchors, ensuring that every factual statement can be traced back to a specific external source.

              1. Optional Citation Insertion

            Some RAG implementations explicitly insert inline citations or footnotes, indicating which document or page each fact originates from. This enhances transparency and credibility, especially in domains like healthcare or legal research.

                1. Outcome

              The final output is a coherent, conversational response that is both fluent and verifiably sourced—reducing the likelihood of “hallucinations”.

              Why does RAG matter?

                  • Accuracy and Currency

                Because RAG fetches fresh data at query time, it can provide up-to-the-minute answers—even if the underlying LLM was last trained months or years ago. For example, a healthcare AI using RAG can retrieve the latest CDC guidelines before generating a recommendation, rather than relying on outdated training data.

                    • Reduced Hallucinations

                  By grounding responses in concrete, external sources, RAG dramatically lowers the risk of fabricated or misleading information. When users see inline citations, trust in AI-generated answers increases.

                      • Domain Specialization

                    Organizations can connect RAG systems to highly specialized knowledge bases—like a law firm’s case archives or a manufacturer’s product specs—without retraining the LLM. The AI becomes an expert in that domain simply by accessing the right repository at query time.

                        • Cost Efficiency

                      Instead of fine-tuning a massive LLM every time new information is added, you update the external datastore. This “decoupling” of model training from content updates is faster, cheaper, and more scalable—especially for companies that produce time-sensitive reports or whitepapers.

                          • Competitive Differentiation

                        As Google’s “AI Mode” is rolled out on a more massive scale, organizations that optimize for RAG-driven visibility gain a strategic edge. Their content is more likely to be surfaced in AI-generated summaries and chatbot answers, capturing traffic that might otherwise bypass static search engine results.

                        How to optimize content for RAG-driven AI search engines?

                        Google EEAT

                        Optimizing for RAG workflows means ensuring your content is structured, authoritative, and easy for retrieval algorithms to pinpoint. Below are actionable tactics:

                        1. Craft Clear, Structured, Answer-Focused Content

                        AI retrieval steps look for self-contained “snippets” that directly match user queries. Use semantic headings for primary sections so AI bots can isolate exact sections to quote. Begin each section with a concise answer.

                        For example:

                        How to File Sales Tax in California (2025 Update)

                        As of June 2025, all California small businesses must file sales tax returns by the 15th of each month. Refer to the California Department of Tax and Fee Administration website for exact forms.

                            • Use bullet lists and numbered steps for procedures to enhance snippet eligibility.

                            • Include a “TL;DR” summary at the top of long articles so RAG systems can grab that concise overview.

                          2. Embed Up-to-Date, Authoritative References

                          RAG systems ground their output in trusted documents. Pages that cite reputable, recent sources—such as government websites, peer-reviewed journals, or industry white papers—signal higher trustworthiness.

                              • Link to the latest guidelines or studies with a clear “Last Updated” date.

                              • Regularly audit and update publication dates to maintain freshness, benefiting both human readers and AI bots.

                            Example:
                             “According to the CDC’s May 2025 update on COVID-19 guidelines, mask mandates for healthcare workers in high-risk settings remain in effect (CDC, May 2025).”

                            3. Use Conversational Phrasing and Natural-Language Keywords

                            RAG retrieval often relies on vector-based similarity, matching semantic meaning rather than exact keywords. Write headings as questions users would ask—e.g., “What Are the 2025 Tax Deadlines for Freelancers in Texas?”—and follow with an immediate, concise answer.

                                • Include synonyms and related terms, such as “self-employed tax due dates” and “independent contractor tax deadlines,” to create multiple semantic entry points.

                                • Adopt a conversational tone so your content aligns with how AI systems interpret queries, boosting retrieval probability.

                              4. Leverage Schema Markup and FAQ/HowTo Blocks

                              Structured data markup—like FAQ Page or How To schema—helps AI crawlers precisely identify Q&A pairs and step-by-step instructions.

                                  • Wrap each Q&A pair in FAQ Page JSON-LD so RAG systems know these are self-contained snippets.

                                  • Use How To schema for multi-step guides, clearly delineating each step.

                                When Google’s AI Mode or other RAG-enabled platforms crawl your page, they can directly parse these structured blocks without scanning raw text.

                                5. Build Topical Authority and Maintain a Clean Technical Foundation

                                RAG systems prefer content from authoritative domains with strong topical clusters.

                                    • Publish comprehensive guides that interlink subtopics, demonstrating subject-matter depth.

                                    • Acquire backlinks from reputable industry publications—these act as trust signals in both traditional SEO and AI retrieval scoring.

                                    • Optimize technical SEO: ensure fast page load times, mobile responsiveness, secure HTTPS hosting, and accurate XML sitemaps so crawlers can index every relevant page.

                                  Tip: Use tools like Google Search Console to verify your sitemap and crawling status. If pages are excluded, AI retrieval systems won’t be able to find your snippets, regardless of content quality.

                                  6. Monitor and Adapt to AI Search Analytics

                                  Once your content is live, track AI-driven search performance via analytics platforms that show which snippets are being cited in chatbot outputs or AI summaries.

                                      • Review query logs to identify gaps and update content accordingly.

                                      • Refresh your knowledge base and schema markup periodically to keep pace with algorithmic changes.

                                    By treating optimization as an ongoing process rather than a one-time project, you ensure continual visibility in evolving RAG-driven ecosystems.

                                    7. Incorporate Off-Page SEO And PR Tactics for AI Visibility

                                    Traditional digital PR often promoted press releases, link-building or aggressive directory submissions. In certain AI search contexts, off-page tactics, like creating press releases or being cited on article directories, can cause RAG systems to index multiple instances of your content, increasing the likelihood of snippet selection.

                                    In my short YouTube video, I demonstrate how these tactics, some of which may be called “spammy”, can boost visibility in AI-based searches by flooding the retrieval index with relevant signals. While this approach carries risks in traditional SERPs, it can yield surprisingly effective results in AI-driven environments—so long as you monitor for negative user feedback or credibility issues.

                                    FAQs

                                    What is the difference between RAG and a standard LLM response?

                                    A standard LLM generates answers based solely on its pretraining data, which may be outdated if trained months ago. RAG, by contrast, performs a real-time search of external documents before generating an answer, ensuring the information is up-to-date and grounded in factual sources.

                                    Can I use RAG to search proprietary company files?

                                    Yes. By connecting a RAG-enabled system to your internal knowledge base—such as a SharePoint repository or a private document store—your organization can get highly specialized answers rooted in proprietary data without retraining the entire model.

                                    How do schema markup and structured data help AI retrieval?

                                    Schema markup like FAQ Page or How To tells AI crawlers exactly where Q&A pairs and step-by-step instructions begin and end, so retrieval engines can extract self-contained snippets without scanning the entire page. This increases the chances of your content being quoted verbatim in AI-generated summaries.

                                    Checklist

                                        • Identify and segment core Q&A snippets with clear semantic headings.

                                        • Embed date-stamped, authoritative citations (e.g., government or peer-reviewed).

                                        • Use conversational, question-style headings and varied synonyms.

                                        • Apply FAQ Page or How To schema markup around structured content.

                                        • Ensure fast load times, mobile optimization, and valid XML sitemaps.

                                        • Monitor AI search analytics to track snippet performance and update.

                                        • Experiment with off-page snippet postings; measure AI retrieval impact.

                                      Brief Summary and Conclusion

                                      Modern AI search engines and LLMs harness RAG workflows to merge external data retrieval with text generation, often producing answers that are highly accurate and current. By structuring content with clear semantic headings, embedding up-to-date citations, using natural-language Q&A phrasing, and applying FAQ Page or How To schema, you make it easier for AI retrieval to spot—and quote—your material without resorting to a virtual game of hide-and-seek. 

                                      Building topical authority, maintaining strong technical SEO, and even testing off-page snippet tactics can further boost your visibility in AI-driven searches. As AI search evolves, continually monitoring and adapting your strategy will be crucial for long-term success in the RAG-powered landscape.

                                      How do you optimize for Google’s new AI‑Mode answer summaries?

                                      Caleb Turner on May 28, 2025

                                      Google now runs two separate generative‑AI surfaces inside Search: AI Overviews (a quick snapshot embedded in the classic results page) and AI‑Mode (a standalone, Gemini‑powered tab that behaves more like a research assistant). To earn citations in either, you still need strong ranking signals, iron‑clad E‑E‑A‑T and snippet‑ready prose, yet the tactics differ enough that you must optimise for both layers.

                                      Key Takeaways

                                      • AI Overviews ≠ AI‑Mode. Overviews are inline snapshots; AI‑Mode is an opt‑in, dedicated search mode with deeper follow‑ups.
                                      • Overviews appear automatically when Google’s systems deem a query complex enough and safe; AI‑Mode is user‑initiated via a new AI tab.
                                      • Ranking top‑10 still matters—Overviews pull from high‑ranking, verified documents first.
                                      • Put a 60‑–80‑word hero answer under every H1 to maximise extractability.
                                      • E‑E‑A‑T + freshness remains the admission ticket for both layers.
                                      • Expect CTR to fall on Overview queries; offset with branding and lead magnets.

                                      Detailed Guide

                                      1. How do AI Overviews and AI‑Mode actually differ in 2025?

                                      Feature AI Overviews (inline) AI‑Mode (standalone)
                                      Launch timeline US rollout May 14 2024 → 100+ countries Oct 2024  US mass rollout May 20 2025 after Labs testing 
                                      Interface Appears above organic links inside standard SERP; collapsible; cites sources as chips Separate AI tab or toggle; full‑screen conversational UI; shows citations plus follow‑up prompts
                                      Use‑case Quick snapshot for moderately complex “how/why” questions Deep research, multi‑step planning, agentic tasks (e.g., buying tickets, data comparisons)
                                      Trigger Automatic—requires query to meet content‑safety + complexity thresholds Manual—user selects AI‑Mode; no popularity threshold
                                      Model Gemini 2.x tuned for latency Custom Gemini 2.5 with query fan‑out + Deep Search

                                      Why it matters: Overviews reward concise clarity; AI‑Mode rewards depth and interactivity.Optimise pages to satisfy both in one pass: lead with a distilled answer, then dive deep.

                                      How do AI Overviews and AI‑Mode actually differ in 2025

                                      2. When does Google show an AI Overview?

                                      Google has never published exact numbers, but data from SE Ranking and Search Engine Land suggest that queries need both sufficient search volume and a level of informational complexity.

                                      Guideline: Pages that already rank for queries with ≥100 monthly US impressions and 8‑plus words are far more likely to trigger an Overview.

                                      What This Means for SEO & Content Strategy

                                      SEO Moves

                                      1. Track impression‑heavy question keywords in Search Console.
                                      2. Consolidate overlapping articles—one URL per FAQ.
                                      3. Refresh answers quarterly to keep Overview eligibility.

                                      3. Crafting the hero answer—your 80‑word golden ticket

                                      A well‑formed hero paragraph can surface in both Overviews and the first AI‑Mode answer.

                                      • Length: 60–80 words, two sentences max.
                                      • Structure: statement → key fact → source cue (stat/name).
                                      • Branding: mention brand once in first clause.
                                      • Location: immediately after H1, above any images or ads.

                                      Copy hack: Draft two variants (60 w & 80 w) and alternate every 14 days to compare CTR. 

                                      4. Two‑phase summarisation still underpins both layers

                                      Google’s 2024 patent describes an espresso (fast) and slow‑brew (deep) retrieval loop Overviews rely mostly on espresso; AI‑Mode can wait for slow‑brew and even expand with Deep Search, issuing hundreds of sub‑queries.

                                      5. Verification signals—earning the invite

                                      Both systems filter the candidate set to verified documents before prompting the model. Signals include:

                                      1. Authorship credentials with professional links.
                                      2. Citations to primary research (government, peer‑reviewed, corporate filings).
                                      3. Structured data—Article, FAQPage, HowTo, FactCheck.
                                      4. Fresh timestamps and frequent updates for YMYL topics.
                                      5. Fast Core Web Vitals (LCP < 2.5 s; INP < 200 ms).

                                      6. Snippet engineering—teaching robots to skim

                                      Robots skim like distracted humans. Help them:

                                      • ≤ 3‑sentence paragraphs; no walls of text.
                                      • Bullet or numbered lists for steps.
                                      • Definition call‑outs (> blockquote or styled div).
                                      • Question‑form headings to mirror Google’s reformulation: “How does…”

                                      7. Technical hygiene—speed still kills eligibility

                                      Even the smartest model aborts slow pages:

                                      • LCP < 2.5 s (espresso cut‑off).
                                      • INP < 200 ms.
                                      • Serve images in AVIF/WebP and lazy‑load below the fold.

                                      8. Branding inside the snippet—CTR insurance

                                      Because Overviews often satisfy intent without a click, brand recall is your safety net:

                                      1. Put brand in the first 50 characters of <title>.
                                      2. Use a distinctive favicon.
                                      3. Embed a next‑step teaser (“Download the checklist”) below the hero paragraph.

                                      9. Measuring success across both layers

                                      Metric Overviews AI‑Mode Target
                                      Impressions vs. Clicks ▼ CTR N/A* Monitor 30‑day delta
                                      Branded search volume ↑ if citations recall brand ↑ via deeper engagement +5 % YoY
                                      Scroll depth & dwell time Standard Longer sessions ≥ 90 s
                                      Assisted conversions Post‑click purchases Research assist → return Attribute multi‑touch

                                      *AI‑Mode traffic logs separately in Search Console’s AI tab (beta).

                                      10. Pitfalls to avoid

                                      • Burying answers under anecdotes
                                      • Splitting one FAQ across multiple URLs
                                      • Out‑of‑date stats (Overviews drop stale pages fast)
                                      • Ignoring long‑tail queries that still deliver clean clicks

                                      Example / Template

                                      <!– 74‑word hero snippet under H1 –>

                                      <p>Google’s AI‑Mode shows a fully cited answer in its dedicated tab, while AI Overviews

                                      surfaces a concise snapshot above organic links. Rank in the top‑10, write a

                                      60–80‑word solution here, and back it with expert citations to earn both source

                                      chips.</p>

                                      FAQs

                                      Will AI‑Mode kill my CTR?

                                      AI‑Mode sits behind a tab, so only sessions where users opt in bypass organic links entirely. AI Overviews is the bigger CTR threat, trimming clicks by 10‑25 % on affected queries. Mitigate via branded teasers and interactive assets.

                                      Is schema markup still worth the effort?

                                      Yes—FAQPage, HowTo, and FactCheck schema mirror the AI layers’ Q&A structure, accelerate verification, and can trigger rich snippets when no AI answer shows.

                                      Does AI‑Mode penalise affiliate sites?

                                      No direct penalty, but thin, boiler‑plate reviews rarely count as verified. Add first‑hand photos, test data and disclosure labels.

                                      Can I opt out of AI answers?

                                      No. Blocking Googlebot removes you from Search entirely. Instead, lean in—optimise hero snippets, strengthen branding, and turn AI citations into authority signals.

                                      Ten‑Point Action Checklist

                                      • Audit recurring FAQs and rankings.
                                      • Write 60‑80‑word hero paragraphs.
                                      • Add author credentials, citations, fact‑check schema.
                                      • Use question‑form H2/H3s.
                                      • Break answers into ≤ 3 sentences & lists.
                                      • Hit LCP < 2.5 s, INP < 200 ms.
                                      • Build expert backlinks.
                                      • Monitor AI citations and AI‑Mode sessions.
                                      • Refresh content quarterly (monthly for YMYL).
                                      • Track impressions, brand queries, conversions.