What is robots.txt?
A robots.txt file tells search engine crawlers which pages they can and cannot access on your website. Place it at the root of your domain (e.g. example.com/robots.txt). This tool also lets you block AI training crawlers like GPTBot and ChatGPT.