Free Tool
Robots.txt Generator
Build a robots.txt file with crawler-specific access rules, crawl-delay settings, restricted directories, and sitemap directives.
Generator Settings
Configure default and crawler-specific directives
Search Robots
Restricted Directories
Paths are relative to root and typically end with a trailing slash.
Generated Robots.txt
Click create to generate your file, then copy or download it and place it at your site root as /robots.txt.
Need help validating your technical setup? Request a free SEO review.