Robots.txt Generator

Configure your robots.txt rules. Copy the output to your website root.

USER-AGENT

User-agent (leave * for all bots)

RULES

Allow all paths (/)

Disallow paths (one per line)

OPTIONS

Sitemap URL

Crawl-delay (seconds, 0 = none)

Block AI crawlers (GPTBot, Google-Extended, CCBot, etc.)

OUTPUT