What is indexing?

Indexing, explained

Google describes Search in three stages. Crawling: automated programs download pages they find. Indexing: Google analyzes the text, images and video on a page and stores the information in the Google index. Serving: when someone searches, Google returns results from that index. Each stage is a filter. Plenty of pages get crawled and never make it to the index.

During indexing Google also groups duplicate and near-duplicate pages and picks one canonical to represent the group. The others are known but not served. That's why a page can be "crawled" and "excluded" at the same time in Search Console.

Google is explicit that none of this is guaranteed. Its docs say it doesn't guarantee it will crawl, index or serve your page, even if it follows the Search Essentials, and that not every page Google processes will be indexed. Whether a page makes it in depends on its quality, how unique it is, and how the rest of your site looks.

To check a page, use the URL Inspection tool in Search Console. It tells you whether the URL is on Google, which canonical Google chose, and when it was last crawled. For the whole site, the Page indexing report groups URLs by status: indexed, excluded by noindex, duplicate, crawled but not indexed, and so on.

Getting indexed faster is mostly about the basics: link to new pages from pages Google already visits, list them in your sitemap, make them clearly different from what you already have, and don't publish faster than the site's reputation supports. Requesting indexing in Search Console helps for a handful of important URLs, not for thousands.

Why it matters for founders

Every SEO win depends on this step. For a new product, the question isn't "how do I rank #1" yet, it's "are my pages in the index at all". Check that first before rewriting content that Google has never shown to anyone.

Example

You publish a comparison page. URL Inspection says "URL is not on Google: Discovered – currently not indexed". You link to it from your homepage and pricing page, and it's crawled and indexed within days.

Common mistakes

• Assuming a published page is indexed. Check with URL Inspection.
• Resubmitting the same URLs over and over instead of fixing why they're excluded.
• Reading "Excluded" as an error. Noindexed and duplicate pages are supposed to be excluded.
• Publishing large batches of similar pages and expecting them all to be indexed.