Crawl-delay: 10 # Block dedicated AI model-training crawlers User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / # START YOAST BLOCK # --------------------------- User-agent: * Disallow: Sitemap: https://spa.com/sitemap_index.xml # --------------------------- # END YOAST BLOCK