# robots.txt — AEO-correct configuration # # DO: explicitly allow AI answer-engine crawlers. Many managed platform # templates and CDN WAF rules block these by default or strip their user # agents before they reach origin. AEO crawler config is NOT the same as # SEO crawler config — different bots, different rules. User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / User-agent: CCBot Allow: / # Everyone else User-agent: * Allow: / Sitemap: https://example.com/sitemap.xml