Robots.txt Generator

Search Engine Crawling

Generate robots.txt files with an easy visual editor — set crawl rules, sitemaps, and user-agent directives with presets

🤖

Presets:

Site URL

Used for sitemap absolute URLs

User Agent Rule

User-agent

Disallow paths

Allow paths

Crawl-delay (seconds)

Sitemaps

Generated robots.txt

User-agent: *
Disallow:

Place this file at the root of your website as robots.txt