How to diagnose a page that is not indexed

Name the stage before changing the page

Discovery means Google knows a URL exists. Crawling means a crawler retrieves it. Indexing means Google processes the page and may include it in its search index. Ranking concerns which results appear for a particular search.

These stages explain why “we submitted the sitemap” and “the page ranks” are different statements. A sitemap lists URLs for discovery. It does not force Google to include every page.

Inspect the exact public address

Open the correct Search Console property and inspect the full URL, including its path. Use the intended public address, not a staging URL. In the URL Inspection report, read the indexing reason, last crawl information and canonical details where available.

The initial report describes Google’s stored information. Run a live test to check the current page. If you just removed a technical block, the stored result can still describe the earlier version. Keep both results and their dates instead of treating them as contradictory.

Match the symptom to a check

Do not remove every restriction automatically. Private pages and staging sites may be intentionally excluded. First establish whether this exact URL is meant to appear in search.

Know the difference between robots.txt and noindex

Robots.txt controls crawling; it is not a reliable way to prevent a URL appearing in search. A noindex instruction controls indexing, but Google must be able to crawl the response to see it.

Worked example · illustrative, not a client result

A public service page inherits a staging noindex tag. The intended correction is to remove that tag from the public page, publish the change, inspect the response and run a live test. Adding the URL to a sitemap without removing the tag leaves the actual problem in place.

Check the correction, then wait for evidence

1. Load the final URL and confirm it returns the intended page.
2. Check crawl permission, indexing instructions and the canonical URL.
3. Link to it from a relevant public page and include it in the sitemap.
4. Run the live inspection and request indexing when appropriate.
5. Record the date and revisit the stored indexing result later.

Repeated requests do not establish progress. Technical eligibility also cannot force selection into the index. Once indexing is confirmed, use search performance data to study exposure and clicks separately.

Continue learning