This domain is for sale.anticrawl.com is available to buy.Make an offer
AntiCrawl
Free tool · Beta

Control which AI crawlers can use your site.

Check your site against known AI crawlers and generate the robots.txt rules to allow or block each one.

Try the tool How it works

Free beta · No signup · Runs in your browser

What it is

Control which AI crawlers can use your site.

AntiCrawl is a free tool that generates robots.txt rules for the current wave of AI crawlers (GPTBot, ClaudeBot, PerplexityBot and more) and checks what your existing robots.txt already allows. It runs entirely in your browser.

Features

What it does

Up-to-date bot list

Covers the AI crawlers from major providers, grouped by training, AI search and user-fetch.

Pick a policy

Block training only, block everything, or fine-tune per bot. Copy the rules out.

Check what you have

Paste your current robots.txt and see which AI bots it allows or blocks today.

Private

No crawl of your site, no account. Everything runs locally.

How it works

Three steps, no setup

  1. 1

    Choose a policy

    Start from a preset, then fine-tune which bots to allow or block.

  2. 2

    Add your paths

    Keep the default of blocking everything, or list specific paths.

  3. 3

    Copy the rules

    Paste the generated block into your robots.txt and deploy.

Try it now · Free beta

Try AntiCrawl

Use it right here. Nothing you enter leaves your browser.

1 Choose a policy

2 Fine-tune per bot

3 Check an existing robots.txt (optional)

Generated

robots.txt rules


    

    
robots.txt is a voluntary standard: reputable crawlers follow it, others may not. User-agent tokens were checked in October 2026; operators add new ones over time. Runs entirely in your browser.

  





  






  

    
FAQ
Questions, answered

    
Is it free?
Yes. AntiCrawl is free to use while it’s in beta, with no account required.


Is my data stored anywhere?
No. Everything runs in your browser. Nothing you enter is sent to a server, and it clears when you reload.


Does blocking a bot in robots.txt actually stop it?
robots.txt is a voluntary standard. Reputable crawlers honour it; others may ignore it. For hard enforcement you also need server or WAF rules, which this tool can point you toward.


Is the bot list guaranteed complete?
No. User-agent tokens were checked in October 2026, but operators add and rename bots over time.