# robots.txt for www.foliasecurity.com # # Search engines, and AI assistants that read a page to answer someone's # question, are welcome. Crawlers that collect pages as training material # for AI models are not. # Everyone not named below (including Googlebot and Bingbot) User-agent: * Allow: / # AI search and answer crawlers: allowed User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User User-agent: DuckAssistBot User-agent: MistralAI-User Allow: / # AI training crawlers: blocked User-agent: GPTBot User-agent: ClaudeBot User-agent: anthropic-ai User-agent: Google-Extended User-agent: Applebot-Extended User-agent: CCBot User-agent: meta-externalagent User-agent: Bytespider User-agent: cohere-ai User-agent: Diffbot Disallow: / Sitemap: https://www.foliasecurity.com/sitemap.xml