How to Check if a URL Is Indexed at Scale

How do you check if a URL is indexed at scale? Use the Google Search Console URL Inspection API, which returns Google's own index verdict for each URL at up to 2,000 queries per day and 600 per minute per verified property. Feed it your list of URLs, read the coverage state for each, and you get a reliable, per-URL answer without checking pages by hand. Avoid the site: operator for this, because Google says it is only an estimate. For a full site, a bulk checker built on the same API is what turns thousands of URLs into a clean indexed or not-indexed report.

Checking one URL is easy: paste it into Search Console and read the result. Checking ten thousand is a different job. Once a catalog, a programmatic set of pages or a large content site grows past what you can inspect by hand, you need a method that is both accurate and repeatable. Here is how to do it right, and which shortcuts to avoid.

Why the site: operator is the wrong tool for this

The instinct is to type site:yourdomain.com and read the number Google shows. Do not build anything on it. Google has stated plainly and repeatedly that site: is an estimate, not a count of indexed pages. It swings between data-center refreshes, misses pages that are genuinely indexed, and sometimes lists URLs that are not really there. For a rough gut check it is fine. As a source of truth for which specific URLs are indexed, it is unreliable, and it gives you no per-URL detail to act on.

The same warning applies to third-party checkers that scrape the site: results or run automated Google searches. They inherit every flaw of the estimate and add the risk of being blocked or rate-limited. If a tool cannot tell you which method it uses, assume it is scraping, and treat its numbers as approximate.

The reliable method: the URL Inspection API

Google provides an official way to check index status at scale, and it is the one every honest bulk checker uses under the hood. The Search Console URL Inspection API returns the same verdict you see in the URL Inspection panel, one URL at a time, programmatically. For each URL it reports the coverage state, the last crawl time, the canonical Google chose, and whether the page is on Google.

The limits matter for planning. Each verified property allows 2,000 URL inspections per day and 600 per minute. That is generous for most sites and the reason a bulk checker batches requests and spreads them across the day for very large properties. The critical detail is that this API only reads status. It reports whether a URL is indexed; it never submits or forces indexing. Checking and submitting are two separate jobs, which is why a good workflow pairs the read with an official submission step.

How to read the results

A per-URL check returns more than yes or no. The states you care about map to specific problems. "Submitted and indexed" is the goal. "Crawled, currently not indexed" means Google fetched the page and chose not to keep it, usually a quality or duplication verdict. "Discovered, currently not indexed" means Google knows the URL but has not crawled it yet, often a crawl-budget signal on a large site. "Excluded by noindex tag" or "blocked by robots.txt" point straight at a mechanical blocker.

Reading the state per URL is what makes a scaled check useful, because it turns a list into a to-do list. A batch of "crawled, currently not indexed" points you at thin or duplicate content. A wall of "discovered, currently not indexed" points at crawl budget and internal linking. When you check thousands of URLs, the pattern in the states tells you the fix faster than any single page would.

Turning a check into a fix

Checking is only half the loop. Once you know which URLs are missing and why, the next step is getting the real ones crawled and reconsidered. That means clearing the mechanical blockers the states expose, making thin pages worth keeping, strengthening internal links, and submitting priority URLs through the official Google Indexing API and Bing IndexNow rather than waiting. A workflow that only checks leaves you with a spreadsheet; a workflow that checks index status and then submits through approved channels closes the loop.

Teams that already pull data programmatically, for instance through a web scraping API that returns clean structured data, can wire the same URL list into a scheduled index check and a submission step, so coverage is monitored continuously instead of audited once a quarter. That is the difference between finding out a section fell out of the index months late and catching it the week it happens.

How often should you check?

For a stable site, a full index check monthly is enough to catch drift. For a site that publishes or changes often, a large ecommerce catalog, a news site, a programmatic set of pages, weekly or continuous monitoring pays off, because pages drop out of the index quietly and traffic follows. The point of checking at scale is not a one-time audit; it is an early-warning system. When a batch of URLs flips from indexed to not, you want to know in days, not after the revenue dip shows up in analytics.

The bottom line

To check indexation at scale, use the URL Inspection API for Google's real per-URL verdict, ignore the site: estimate for anything that matters, and read the coverage states as a map of what to fix. Then pair the check with official submission so a non-indexed URL becomes an action instead of a note. If you would rather not build the pipeline yourself, a bulk index checker that reads the official API and resubmits through approved channels does the whole loop and keeps watching afterward.