1. Sitemaps 0.9 Protocol Rules
Every URL must be an absolute, properly XML-escaped https:// address wrapped in <url><loc>...</loc></url>. Only include canonical, indexable (HTTP 200) pages—exclude redirected, 404, or noindex URLs.
THE TOOLBOX / DATA & TECHNICAL SEO
Turn any list of URLs or relative paths into a clean, valid sitemap.xml file with configurable lastmod, changefreq, and depth-based priority—plus build and test robots.txt crawler rules directly in your browser.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://mysite.test/</loc>
<lastmod>2026-11-22</lastmod>
<changefreq>weekly</changefreq>
<priority>1.0</priority>
</url>
<url>
<loc>https://mysite.test/features/</loc>
<lastmod>2026-11-22</lastmod>
<changefreq>monthly</changefreq>
<priority>0.9</priority>
</url>
<url>
<loc>https://mysite.test/pricing/</loc>
<lastmod>2026-11-22</lastmod>
<changefreq>weekly</changefreq>
<priority>0.9</priority>
</url>
<url>
<loc>https://mysite.test/blog/</loc>
<lastmod>2026-11-22</lastmod>
<changefreq>daily</changefreq>
<priority>0.8</priority>
</url>
<url>
<loc>https://mysite.test/docs/</loc>
<lastmod>2026-11-22</lastmod>
<changefreq>weekly</changefreq>
<priority>0.8</priority>
</url>
<url>
<loc>https://mysite.test/contact/</loc>
<lastmod>2026-11-22</lastmod>
<changefreq>yearly</changefreq>
<priority>0.5</priority>
</url>
</urlset>TECHNICAL SEO & CRAWLABILITY GUIDE
Every URL must be an absolute, properly XML-escaped https:// address wrapped in <url><loc>...</loc></url>. Only include canonical, indexable (HTTP 200) pages—exclude redirected, 404, or noindex URLs.
Crawlers match the most specific User-agent block first (falling back to *). Within a block, * matches any sequence of characters, $ anchors the end of a URL, and the longest matching pattern determines Allow vs. Disallow.
Download sitemap.xml and robots.txt, deploy both to your site's root public directory, reference the sitemap URL inside robots.txt, and submit the sitemap in Google Search Console.
Preview search snippets in the Google SERP Simulator, parse query strings in the URL Inspector, or format structured JSON-LD data in the JSON Formatter.
An XML sitemap (sitemap.xml) is a machine-readable list of your website’s canonical URLs formatted to the Sitemaps 0.9 protocol. Submitting it to Google Search Console and Bing Webmaster Tools helps search engines discover new and updated pages faster—especially on large or newly launched sites.
According to the Sitemaps protocol, a single XML sitemap may contain up to 50,000 URLs and must be no larger than 50 MB uncompressed. Larger sites split URLs across multiple sitemaps referenced from a sitemap index file.
Google actively uses <lastmod> when dates are accurate and consistently reflect meaningful content updates. Google ignores <changefreq> and <priority>, though other crawlers and internal auditing tools still read them—so you can toggle each tag on or off cleanly.
Modern search engines (including Googlebot and Bingbot) use longest-match precedence: the rule with the longest matching path pattern wins. If an Allow and Disallow rule match with the exact same length, Allow takes precedence.
Place both files at the root of your domain—https://yourdomain.com/sitemap.xml and https://yourdomain.com/robots.txt—and include a "Sitemap: https://yourdomain.com/sitemap.xml" directive inside robots.txt.