Skip to main content
ConvertLab
Tools

Web & URLs

robots.txt Analyzer

Paste a robots.txt and check exactly what any user agent may fetch, with the matching rules explained.

  • Runs in your browser
Blocked

The longest matching rule is Disallow /internal/.

Rules in the matched group (*)

Disallow/internal/matches

Crawl-delay for this agent: 5s

Sitemaps declared

  • https://convertlab.example/sitemap.xml

Was this tool useful?

Stored locally in your browser only — not sent anywhere.

Tools you may need

URL Encoder/DecoderPercent-encode text for URLs, or decode an encoded string back to plain text.URL ParserBreak any URL into protocol, host, port, path, query parameters and hash — entirely on-device.Slug GeneratorTurn titles and phrases into clean, URL-friendly slugs.