User-agent: * Disallow: /archives/ # AIs User-agent: Mediatoolkitbot Disallow: / User-agent: CCBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: GPTBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Omgilibot Disallow: / User-Agent: FacebookBot Disallow: / User-Agent: anthropic-ai Disallow: / user-agent: Amazonbot disallow: / # AHrefs User-agent: AhrefsBot Disallow: / # riddler.io User-agent: Riddler Disallow: / # Semrushbot User-agent: SemrushBot-SA Disallow: / # Ntent User-agent: NTENTbot Disallow: / # MJ12 Bot User-agent: MJ12bot Disallow: / User-agent: MojeekBot Disallow: / User-agent: Flamingo_SearchEngine+(+http://www.flamingosearch.com/bot) Disallow: / User-agent: Flamingo_SearchEngine Disallow: / User-Agent: Claudebot Disallow: / User-agent: AdsBot-Google Disallow: / User-agent: AdsBot-Google-Mobile Disallow: / User-agent: AdsBot-Google-Mobile-Apps Disallow: / User-agent: 360Spider Disallow: / User-agent: Flamingo_SearchEngine+(+http://www.flamingosearch.com/bot) Disallow: /