Guide

THE TOOLBOX / DATA & TECHNICAL SEO

XML Sitemap Generator —
Sitemap.xml Builder & Robots.txt Tester.

Turn any list of URLs or relative paths into a clean, valid sitemap.xml file with configurable lastmod, changefreq, and depth-based priority—plus build and test robots.txt crawler rules directly in your browser.

UTF-8 · sitemaps.org 0.9 · RFC 9309
Templates:
#1
#2
#3
#4
#5
#6
Generated sitemap.xml
6 valid URLs · 1.03 KB
✓ Valid Sitemap 0.9 Schema6 / 50,000 URL limit
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://mysite.test/</loc>
    <lastmod>2026-11-22</lastmod>
    <changefreq>weekly</changefreq>
    <priority>1.0</priority>
  </url>
  <url>
    <loc>https://mysite.test/features/</loc>
    <lastmod>2026-11-22</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.9</priority>
  </url>
  <url>
    <loc>https://mysite.test/pricing/</loc>
    <lastmod>2026-11-22</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.9</priority>
  </url>
  <url>
    <loc>https://mysite.test/blog/</loc>
    <lastmod>2026-11-22</lastmod>
    <changefreq>daily</changefreq>
    <priority>0.8</priority>
  </url>
  <url>
    <loc>https://mysite.test/docs/</loc>
    <lastmod>2026-11-22</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.8</priority>
  </url>
  <url>
    <loc>https://mysite.test/contact/</loc>
    <lastmod>2026-11-22</lastmod>
    <changefreq>yearly</changefreq>
    <priority>0.5</priority>
  </url>
</urlset>

TECHNICAL SEO & CRAWLABILITY GUIDE

Best practices for sitemap.xml and robots.txt configuration.

1. Sitemaps 0.9 Protocol Rules

Every URL must be an absolute, properly XML-escaped https:// address wrapped in <url><loc>...</loc></url>. Only include canonical, indexable (HTTP 200) pages—exclude redirected, 404, or noindex URLs.

2. Robots.txt Precedence & Wildcards

Crawlers match the most specific User-agent block first (falling back to *). Within a block, * matches any sequence of characters, $ anchors the end of a URL, and the longest matching pattern determines Allow vs. Disallow.

3. Deploying & Submitting

Download sitemap.xml and robots.txt, deploy both to your site's root public directory, reference the sitemap URL inside robots.txt, and submit the sitemap in Google Search Console.

Frequently Asked Questions

What is an XML sitemap and why does it help SEO?

An XML sitemap (sitemap.xml) is a machine-readable list of your website’s canonical URLs formatted to the Sitemaps 0.9 protocol. Submitting it to Google Search Console and Bing Webmaster Tools helps search engines discover new and updated pages faster—especially on large or newly launched sites.

What are the size and URL limits for a single sitemap.xml file?

According to the Sitemaps protocol, a single XML sitemap may contain up to 50,000 URLs and must be no larger than 50 MB uncompressed. Larger sites split URLs across multiple sitemaps referenced from a sitemap index file.

Does Google still use <lastmod>, <changefreq>, and <priority>?

Google actively uses <lastmod> when dates are accurate and consistently reflect meaningful content updates. Google ignores <changefreq> and <priority>, though other crawlers and internal auditing tools still read them—so you can toggle each tag on or off cleanly.

How does the robots.txt tester decide Allow vs. Disallow when rules overlap?

Modern search engines (including Googlebot and Bingbot) use longest-match precedence: the rule with the longest matching path pattern wins. If an Allow and Disallow rule match with the exact same length, Allow takes precedence.

Where should I place sitemap.xml and robots.txt on my site?

Place both files at the root of your domain—https://yourdomain.com/sitemap.xml and https://yourdomain.com/robots.txt—and include a "Sitemap: https://yourdomain.com/sitemap.xml" directive inside robots.txt.