SEO

Sitemap

A sitemap is a file, usually in XML format, that tells search engines about the URLs on a site and their metadata such as last-modified dates. It helps crawlers discover pages more efficiently, but it does not guarantee indexing.

A sitemap is a file that hands search engines a list of a site's URLs along with their metadata, helping crawlers discover pages.
Listing a URL in a sitemap never guarantees indexing; it is only a hint that supports crawling.
A single sitemap file can hold at most 50,000 URLs and cannot exceed 50MB uncompressed (52,428,800 bytes), and it must be UTF-8 encoded.
When a site exceeds those limits, split the sitemap into multiple files and group them with a sitemap index file.
Submit through Search Console or the Sitemap directive in robots.txt; the legacy ping endpoint was deprecated in 2023.

Overview

A sitemap is a file, typically in XML format, that tells search engines which URLs you want surfaced from your site, together with metadata for each URL such as its last-modified date. Search engines read this file to crawl the site more efficiently, and it is especially useful for discovering new or deep pages when those pages are not well connected to one another. That said, Google states plainly that it does not guarantee every entry in a sitemap will be crawled and indexed, so a sitemap should be understood as a discovery hint rather than a promise of indexing.

According to Google, sitemaps deliver the most value when a site is large (roughly more than 500 pages), when it is new and has few external links, or when it includes a lot of rich media such as video and images. As the standard recommends, a sitemap should list the canonical version of each page's URL.

Formats and limits

Google supports three sitemap formats. XML sitemaps are the most versatile and can be extended to cover images, video, news, and multilingual pages; you can also use RSS, mRSS, and Atom 1.0, or a plain text format with one URL per line.

The standard sets the following limits.

URL count: up to 50,000 per sitemap file
File size: up to 50MB uncompressed (52,428,800 bytes)
Encoding: UTF-8, with all values entity-escaped (for example, & becomes &)
loc length: URLs must begin with a protocol (such as http) and cannot exceed 2,048 characters

The basic structure of an XML sitemap consists of <urlset> as the root, a <url> entry for each item, and the required <loc> tag holding the URL. The <lastmod> tag is optional but recommended when its value is accurate and consistent, while Google ignores the <changefreq> and <priority> values.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/foo.html</loc>
    <lastmod>2022-06-04</lastmod>
  </url>
</urlset>

A large site that exceeds these limits splits its sitemap into multiple files and then creates a sitemap index file to group them. The index file is subject to the same maximums of 50,000 entries and 50MB.

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://www.example.com/sitemap-1.xml</loc>
    <lastmod>2022-06-04</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://www.example.com/sitemap-2.xml</loc>
  </sitemap>
</sitemapindex>

Submission methods

Once you have built a sitemap, there are two main ways to make search engines aware of it.

Search Console: submit the sitemap URL directly in the Sitemaps report and review its processing status and any errors.
robots.txt: add a Sitemap directive with an absolute URL so that any search engine reading the file discovers its location.
```
Sitemap: https://www.example.com/sitemap.xml
```

The HTTP ping endpoint that was once used (https://www.google.com/ping?sitemap=...) was announced for deprecation by Google in June 2023 and stopped working six months after that announcement. Google explained that most unauthenticated sitemap submissions turned out to be spam and were not useful, so today only Search Console and robots.txt submission are recommended.

Authoritative basis

The limits and formats above derive from the sitemap protocol 0.9 standard published by sitemaps.org and from the official Google Search Central documentation. The sitemaps.org protocol specifies that a sitemap file must contain no more than 50,000 URLs and must not exceed 50MB uncompressed (52,428,800 bytes), and Google's documentation cites the same figures. The ping deprecation was officially announced on the Google Search Central blog in June 2023 in the post titled "Sitemaps ping endpoint is going away."

Implementation checklist

Include only the canonical URLs you want indexed, and exclude redirected, noindexed, or blocked URLs.
Save the file as UTF-8 and entity-escape special characters.
When you approach 50,000 URLs or 50MB, split the file and group it with a sitemap index.
Populate <lastmod> accurately only when the value is trustworthy; otherwise omit it.
Add a Sitemap: directive to robots.txt and also submit through Search Console.
Remove or ignore any existing code or plugins that still rely on the ping endpoint.
Regularly check read status and errors in the Search Console Sitemaps report.