# robots.txt for https://roguetreasure.com # Search engines and AI search/citation crawlers may crawl the whole public site. # Only the 404 page is kept out of indexes. The site has no private routes, APIs or admin areas. # # AI-readable summary of the site (llmstxt.org): https://roguetreasure.com/llms.txt # Full site content as Markdown: https://roguetreasure.com/llms-full.txt # Search engines User-agent: Googlebot User-agent: Bingbot User-agent: DuckDuckBot User-agent: Applebot Allow: / Disallow: /404.html # AI search, citation and assistant crawlers User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: ClaudeBot User-agent: Claude-SearchBot User-agent: Claude-User User-agent: Claude-Web User-agent: PerplexityBot User-agent: Perplexity-User User-agent: Google-Extended User-agent: Applebot-Extended Allow: / Disallow: /404.html # Everyone else User-agent: * Allow: / Disallow: /404.html Sitemap: https://roguetreasure.com/sitemap.xml