Technical SEO for AI Search Engines: The 2026 Checklist

AI search engines rely on the same web Google has always crawled. That makes technical SEO more important now, not less. The fundamentals. Crawl access, rendering, schema, internal linking, performance. Gate everything downstream. AI engines also add a few signals of their own, like llms.txt and AI bot user-agents, that most sites have wrong by default.

This guide is a focused, ordered checklist of the technical SEO work that matters for AI search in 2026. Written for technical SEOs, dev teams, and anyone responsible for shipping the foundation under an AI SEO program.

What technical SEO changed for AI engines

Three things shifted. Everything else is a tightening of habits that were already good practice.

New bots to allow (and accidentally block)

OpenAI, Anthropic, Perplexity, Google, Microsoft, and Apple all operate their own crawlers with distinct user-agents. Default firewall rules, security plugins, and old robots.txt entries often block them silently. The first technical fix on most engagements is unblocking AI bots.

Schema becomes about retrieval, not rich snippets

Schema markup used to be valued mainly for SERP features. AI engines use it more fundamentally, for entity disambiguation, author attribution, source grounding, and content classification. The most useful schema types overlap with classical SEO, but the completeness expected is higher.

llms.txt as a new positive signal

llms.txt is a relatively new file format that gives AI engines a curated map of your priority content. It is not a directive in the way robots.txt is, and it does not guarantee anything. But the signal appears to be real, and the work to publish it is small.

AI bots: which to allow and how

The major AI bot user-agents to know about today:

• GPTBot. OpenAI's training corpus crawler.
• ChatGPT-User. OpenAI's user-initiated browsing crawler.
• OAI-SearchBot. OpenAI's search-engine crawler.
• ClaudeBot. Anthropic's training and retrieval crawler.
• Claude-Web. Anthropic's browsing user-agent.
• PerplexityBot. Perplexity's crawler.
• Perplexity-User. Perplexity's user-initiated fetcher.
• Google-Extended. Google's opt-in flag for Gemini and AI training. Allowing it does not affect classical Google rankings.
• CCBot. Common Crawl, used as a training source by many open models.
• Applebot-Extended. Apple's opt-in flag for AI features.

For most sites, you want all of these allowed. Your robots.txt should explicitly Allow them rather than relying on defaults. Many WAFs and CDNs block unfamiliar user-agents. Most brands should explicitly test that each bot can fetch a sample of their pages.

Block individual bots only when there is a real reason . unsolicited training of proprietary content, for example. Even then, leave retrieval-time bots (Claude-Web, ChatGPT-User, Perplexity-User) allowed where possible. Blocking them blocks live citations.

Rendering and content visibility

Many AI bots have limited or no JavaScript execution. Content rendered only in client-side JS may be invisible to them. The rendering question is the second technical issue we audit on every engagement.

Recommended approach

• Server-side rendering or static generation. For marketing pages, landing pages, blog content, and documentation, render server-side. AI bots see content immediately, classical search benefits, and performance improves.
• Critical content above the fold without JS. If you cannot SSR everything, at least make sure the page's title, primary heading, direct answer, and main content render without JavaScript.
• Avoid lazy-loaded primary content. Lazy-loading decorative images is fine. Lazy-loading the article body or core product information is not.
• Test with bot-friendly tooling. Use curl/headless requests with each bot's user-agent to confirm the rendered HTML actually contains your content.

Schema that actually matters

Schema is one of the highest-leverage technical SEO investments for AI search. The relevant types overlap with classical SEO but get weighted differently. Priority schema types:

• Organization. The brand entity. Include name, url, logo, sameAs (LinkedIn, Wikipedia, Wikidata, Crunchbase), and address if relevant.
• Article and BlogPosting. All editorial pages. Include author, datePublished, dateModified, publisher, headline, and image.
• Product, Service, SoftwareApplication. Commercial pages. Include name, description, brand, offers, and image.
• Person. Author pages and key team members. Include name, jobTitle, worksFor, and sameAs links to public profiles.
• FAQ. Pages with structured question-answer content. AI engines extract FAQ schema directly into answers.
• HowTo. Pages with step-by-step instructions. High value for instructional queries.
• BreadcrumbList. Helps engines understand page hierarchy.
• Review and AggregateRating. Where genuinely applicable. Do not fake.

Two practices matter as much as which schemas you add:

1. Be complete, not minimal. A half-filled Article schema is worse than one with every relevant field populated.
2. Validate continuously. Run schema validation in CI. Broken schema is worse than no schema for AI engines that try to use it.

llms.txt: what it is and how to use it

llms.txt is a plain-text file you publish at the root of your domain (for example, example.com/llms.txt). It gives AI engines a curated map of your most important content with optional descriptions.

Unlike robots.txt, llms.txt is not a directive. It is an invitation . here are the pages we want AI engines to find first. Engines do not have to honor it, but the signal appears to help, and the work to publish a useful llms.txt is minimal.

What to include

• A short brand description and what you do.
• A curated list of your priority pages. Homepage, key product pages, top blog content, documentation.
• Optional short descriptions for each page explaining what it covers.
• A note about how AI engines should use the content (cite, link, attribute).

What not to do

• Do not dump your sitemap. llms.txt should be curated, not exhaustive.
• Do not use it as a substitute for clean site structure. A page that is hard to find from your homepage is also hard for engines to trust.
• Do not include marketing fluff. Be direct.

Internal linking and topical clusters

Internal linking is one of the highest-leverage SEO investments . and one of the most underused. It directly shapes how AI engines understand the relationships between your pages and the topics they cover.

What strong internal linking looks like

• Topical clusters. A pillar page on each major topic, supported by sub-pages on specific sub-questions. Each pillar links to its sub-pages; each sub-page links back to the pillar.
• Contextual links. Links inside the article body, not just in navigation or footer. Use descriptive anchor text that matches the destination's topic.
• Cross-cluster linking where relevant. When two topics overlap, link the related pages. This signals topical breadth without diluting depth.
• Sane crawl depth. Every important page should be reachable from the homepage in three clicks or fewer.
• Working links. Broken internal links and redirect chains erode trust signals. Audit quarterly.

For more on topical clusters and entity depth, see entity SEO for AI search.

Freshness signals that move citation share

AI engines weight freshness heavily, especially for time-sensitive queries. Pages that look untouched for two years lose citation share quickly. The freshness signals that matter:

• Visible published and updated dates. Include both on the page, in human-readable format and in schema (datePublished, dateModified).
• Real refreshes, not date-bumps. Updating only the date without changing the content is detected and discounted. Quarterly refreshes should include material edits.
• Fresh citations. If you cite external sources, keep them recent. Pages with primary citations from five years ago lose freshness signal.
• Active content cadence. Sites with regular publication signal an active, current source. AI engines weight this at the domain level, not just per page.
• Last-modified HTTP headers. Set them correctly. Many CDNs override them; check.

Performance and Core Web Vitals

Performance affects classical SEO and is even more relevant for AI search. Slow pages get deprioritized in retrieval, which means they never get into the citation set in the first place. The targets are the same ones Google publishes:

• LCP (Largest Contentful Paint). Under 2.5 seconds.
• INP (Interaction to Next Paint). Under 200 ms.
• CLS (Cumulative Layout Shift). Under 0.1.

The fastest performance fixes for most sites are image optimization, avoiding render-blocking JavaScript, and aggressive caching. CDN configuration matters more than most teams expect, many CDNs produce slower pages than necessary because of default cache policies.

The full technical SEO checklist

A consolidated list to work through in order.

1. Robots.txt explicitly allows GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, ChatGPT-User, Perplexity-User, Claude-Web, CCBot, Applebot-Extended.
2. WAF and CDN rules do not silently block AI bots.
3. All critical content is server-rendered or statically generated.
4. Site rendered without JS contains title, headings, primary answer, and body text.
5. Organization schema on the homepage with sameAs links to verified profiles.
6. Article schema on every editorial page with author, dates, and publisher.
7. Product or Service schema on every commercial page.
8. Person schema on author pages and team members.
9. FAQ and HowTo schema where applicable.
10. BreadcrumbList schema across the site.
11. Schema validates in Google Rich Results Test with no errors.
12. llms.txt is published, curated, and current.
13. Topical clusters in place with pillar pages and bidirectional internal links.
14. Every important page reachable from the homepage in 3 clicks or fewer.
15. No broken internal links or redirect chains over two hops.
16. Visible published and updated dates on all editorial pages.
17. Refresh cadence in place for top pages (quarterly, with material edits).
18. Last-modified HTTP headers set correctly.
19. LCP under 2.5s, INP under 200ms, CLS under 0.1 on top pages.
20. XML sitemap submitted and current.
21. Canonical URLs set correctly across the site.
22. Hreflang configured for multilingual sites.

Technical SEO for AI engines is not glamorous. The fixes are unsexy. The wins are quiet. But every other AI SEO investment . Content, authority, entity. Rests on this foundation. Ship the checklist above, validate it monthly, and the rest of the program starts working much faster than most teams expect.