All articles · Technical SEO

XML Sitemaps That Help: What to Include and What lastmod Means

By PerfoMetrix Team·November 5, 2026·5 min read

Most sites have an XML sitemap because a plugin or framework generated one. Far fewer have a sitemap that actually helps. The typical generated file lists redirected URLs, pages marked noindex, parameter variants, and a lastmod that changes on every request.

A sitemap is a list of URLs you’d like search engines to know about, plus a little metadata. It doesn’t force anything into the index, and a bad one rarely causes an outright penalty. But it does shape what crawlers spend time on, and a noisy one teaches them to ignore its signals. This article covers what to include, what the fields mean in practice, and how to check the result.

What a sitemap is for

Search engines discover pages mainly by following links. A sitemap is a second channel, and it’s most useful when links alone are slow or unreliable:

• large sites, where new or deep pages may take a long time to be found through navigation;
• new sites with few external links;
• pages that are poorly linked internally, such as older articles or long-tail product pages;
• sites with frequent changes, where you want updated pages recrawled sooner.

On a small, well-linked site the sitemap matters less, but it costs almost nothing to get right, and Search Console’s sitemap report is a handy view of how many of your pages are indexed.

The format, briefly

The protocol is defined at sitemaps.org and is supported by Google, Bing and others. A minimal file:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/guides/image-delivery</loc>
    <lastmod>2026-10-14</lastmod>
  </url>
  <url>
    <loc>https://example.com/pricing</loc>
    <lastmod>2026-09-02T08:30:00+00:00</lastmod>
  </url>
</urlset>

A few rules that generators sometimes break:

• The file must be UTF-8, and URLs must be absolute, including the protocol.
• Special characters in URLs must be XML-escaped. A query string with & has to be written as &, or the whole file fails to parse.
• One file may hold at most 50,000 URLs and be at most 50 MB uncompressed. Gzip is allowed to save bandwidth, but the limit applies to the uncompressed size.

Include only canonical, indexable URLs

The single most useful rule: every URL in the sitemap should be one you want to appear in search results, exactly as you want it to appear. In practice, that means each URL:

• returns 200 directly, not a redirect and not a 404 or 410;
• is not marked noindex, by meta tag or header;
• is not blocked by robots.txt;
• is the canonical version of the page, with the same protocol, host, trailing slash style and no tracking parameters.

Listing a URL in a sitemap is a hint that it’s the preferred version, so a sitemap full of non-canonical variants contradicts your own canonical tags. Redirected and noindex URLs waste crawl requests and make Search Console’s report harder to read, because the “submitted but not indexed” count fills with pages you never wanted indexed.

What to leave out: internal search results, filtered and sorted listing variants, paginated duplicates you’ve consolidated, login and account pages, thank-you pages, and anything behind authentication.

priority and changefreq: skip them

The protocol defines two optional fields, <priority> and <changefreq>. Google has said it ignores both, and has documented that in its guide to building sitemaps. Most sites set every page to priority 0.5 and changefreq daily anyway, which carries no information.

Removing them makes the file smaller and easier to audit. If another search engine you care about documents a use for them, set them deliberately; otherwise leave them out.

What lastmod means, and how to keep it honest

<lastmod> is the one optional field Google does use, with a condition: it uses the value when it’s consistently and verifiably accurate. If your lastmod values turn out to be unreliable, Google learns to disregard them for your site.

“Last modified” should mean the last time the main content of the page changed in a way that matters to someone reading it:

• a correction or rewrite of the text, a price or availability change, new sections, new structured data that reflects visible content.

It should not change because:

• the sitemap was regenerated, or the page was rendered;
• a copyright year in the footer ticked over;
• a sidebar, related-posts widget or ad slot changed;
• the CSS or JavaScript bundle was redeployed.

The common anti-pattern is lastmod set to the current time on every request. It tells crawlers that every page changes constantly, which is easy for them to verify as false.

The robust approach is to store a content timestamp next to the content itself, updated only by edits to the fields that matter, and to output it in W3C Datetime format: a date (2026-10-14) or a full timestamp with a time zone (2026-10-14T09:00:00+00:00). If your CMS can’t tell a real edit from a cosmetic one, a date-only value that changes on publish and on substantive edits is better than a precise one that changes for every reason.

Splitting large sitemaps

Once you’re near 50,000 URLs, or simply want a better view in Search Console, split the sitemap and list the parts in a sitemap index:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://example.com/sitemaps/products-1.xml</loc>
    <lastmod>2026-10-28</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://example.com/sitemaps/guides.xml</loc>
    <lastmod>2026-10-14</lastmod>
  </sitemap>
</sitemapindex>

Split by page type rather than alphabetically or at random: products, categories, articles, static pages. Search Console then reports indexing per file, and a sudden drop in indexed products stands out instead of being diluted by thousands of articles. The index’s own lastmod should be the newest lastmod within that child file.

Sitemaps can also carry extensions for images, videos and news, and xhtml:link entries for hreflang alternates. They’re worth using when they replace something harder to maintain, but they follow the same rule: only canonical, indexable URLs.

Telling search engines where it is

Two ways, and you can use both:

1. A Sitemap: line in robots.txt with the absolute URL. Any crawler that reads your robots.txt finds it, not just Google.
2. Submitting it in Google Search Console and Bing Webmaster Tools, which also gives you the processing status and error reports.

Google retired its anonymous sitemap “ping” endpoint in 2023, so scripts that still call it achieve nothing. Bing and some other engines support IndexNow for pushing changed URLs; Google doesn’t use it. For Google, an accurate lastmod in a sitemap it already knows is the supported way to signal changes.

Checking what you have

A sitemap review takes less time than most audits:

• Fetch it and confirm it returns 200 with an XML content type and parses cleanly. One stray unescaped & breaks everything after it.
• Sample the URLs: request a few hundred and count anything that isn’t a direct 200, or that carries noindex, or whose canonical points elsewhere.
• Compare lastmod with the dates you’d expect. If every URL shows today, the generator is stamping render time.
• Read the Search Console sitemap report: compare discovered URLs with the page indexing report filtered to that sitemap. A large gap often points to a template or section problem.
• Look for what’s missing: important sections that aren’t listed at all, often because a new content type was added after the generator was configured.

Our XML sitemap guide lists the basic requirements. The overall goal is simple: a sitemap that crawlers can trust, because every URL in it is one you want indexed and every date in it means something.

Tags:XML sitemaplastmodcrawlingindexingSearch Console