Research library

Google’s crawl budget guide says it only matters for very large sites

Strong evidence

Official documentation, sworn testimony, or a large controlled test that others have replicated.

Researcher
Google Search Central
Published by
Google
Date
1 January 2023
Method
official documentation
Sample
Google Search Central documentation (Large site owner's guide to managing crawl budget)
Original
developers.google.com →

What they did

Google published a guide on managing crawl budget for large sites. It sets out which sites should read it and which can skip it entirely.

What they found

The guide tells most site owners not to read it. It aims the advice at sites with more than a million pages that change weekly, or sites over 10,000 pages that change daily. Google calls the page counts a rough estimate, not exact thresholds.

How much weight to give it

This is first-party documentation on developers.google.com, so it carries weight as a statement of how Google approaches crawling. For small and medium sites it is a clear steer that crawl budget is rarely the bottleneck.

What it doesn't prove

It describes crawling, not ranking, so a healthy crawl budget won't lift positions on its own. The thresholds are approximate and the page is revised over time.

Ranking systems

Spotted an error, or did we grade this unfairly? Tell us and we'll amend the entry: use the contact form.