What is the Crawl Stats report in Search Console?

Crawl Stats report, explained

You'll find it under Settings in Search Console, not in the main menu. Google describes it as a report aimed at advanced users, and notes that sites with fewer than 1,000 pages typically don't need this level of detail. It's only available for root-level properties: a Domain property or a URL-prefix property at the root of the site.

The top chart shows total crawl requests, total download size and average response time. Host status tells you whether Google had trouble reaching your site, with separate checks for robots.txt fetching, DNS resolution and server connectivity. Below that, requests are grouped by response code (200, 301, 404, 5xx), by file type (HTML, JavaScript, images, CSS), by purpose (discovery of new URLs or refresh of known ones) and by Googlebot type (smartphone, desktop, image and others).

The useful patterns: a rising average response time usually precedes a drop in crawl requests, because Google backs off slow servers. A large share of 404 or redirect responses points to stale internal links or sitemap entries. Very little "discovery" crawling while you're publishing new pages means Google isn't finding them through links or sitemaps.

It's a summary, not a log. For per-URL detail, use URL Inspection or your server logs. Check it after big changes, like a migration, a new CDN or a large batch of new pages, rather than every week.

Why it matters for founders

If you're publishing many pages and indexing lags, this report shows whether the bottleneck is your server, wasted crawls or discovery. It's the closest Google gets to showing you crawl budget.

Example

After a CMS change, average response time doubles and total crawl requests fall by half over two weeks. You add caching to your page templates, response time drops, and crawling recovers.

Common mistakes

• Looking for it in the main menu instead of Settings.
• Obsessing over it on a 100-page site.
• Ignoring host status warnings about robots.txt fetch failures.
• Reading it on a URL-prefix property that isn't at the root.