Why `site:yourdomain.com` is not a reliable website indexing checker
Google Search Central has repeatedly documented that the `site:` search query operator is not designed for exhaustive index auditing:
- False negatives: A URL that is indexed and serving impressions in Search Console can occasionally fail to appear in a `site:https://yourdomain.com/exact-slug` query.
- Zero diagnostic reason: Even when `site:` returns zero results, it cannot tell you whether the page is blocked by `robots.txt`, marked `noindex`, canonicalized to another URL, or sitting in *Discovered – currently not indexed*.
- No `lastCrawlTime` or canonical comparison: You cannot see when Googlebot last visited the page or which canonical URL Google selected.
The 7 fields that matter in the Google Search Console URL Inspection API
When you connect your Google Search Console property in Searchbop, the desktop worker calls `https://searchconsole.googleapis.com/v1/urlInspection/index:inspect` for your sitemap and audited URLs and extracts seven critical fields from `inspectionResult.indexStatusResult`:
| API Field | Example Value | What It Tells You |
|---|---|---|
verdict |
PASS | NEUTRAL | FAIL |
Whether the URL is indexed and eligible to appear in Google Search (`PASS`), excluded (`NEUTRAL`), or blocked by an error (`FAIL`). |
coverageState |
"Submitted and indexed" / "Crawled - currently not indexed" |
The exact human-readable coverage bucket shown in the Search Console UI. |
indexingState |
INDEXING_ALLOWED | BLOCKED_BY_META_TAG | BLOCKED_BY_HTTP_HEADER |
Immediately reveals whether a <meta name="robots" content="noindex"> or `X-Robots-Tag` blocked indexing. |
robotsTxtState |
ALLOWED | DISALLOWED |
Whether your `robots.txt` allowed Googlebot to fetch the URL at crawl time. |
pageFetchState |
SUCCESSFUL | SOFT_404 | NOT_FOUND | SERVER_ERROR |
Whether your server returned a healthy `200 OK` page or failed during Googlebot’s fetch. |
lastCrawlTime |
"2026-10-30T08:19:44Z" (or omitted) |
Proves whether Googlebot has fetched the latest version of your page since your last deployment. |
userCanonical vs. googleCanonical |
https://example.com/a/ vs. https://example.com/b/ |
Reveals when Google ignored your <link rel="canonical"> tag and folded your page into a different URL. |
Spotting when `googleCanonical` overrides your `userCanonical`
One of the highest-value checks in Searchbop’s indexing audit is comparing `userCanonical` (the <link rel="canonical"> tag you declared in your HTML) against `googleCanonical` (the URL Google actually chose to index). If those two URLs differ, your page is excluded under *Duplicate, Google chose different canonical than user*—usually because internal links, sitemap entries, or near-identical page copy convinced Google that another URL is the primary version.
{
"inspectionResult": {
"indexStatusResult": {
"verdict": "NEUTRAL",
"coverageState": "Duplicate, Google chose different canonical than user",
"robotsTxtState": "ALLOWED",
"indexingState": "INDEXING_ALLOWED",
"lastCrawlTime": "2026-10-25T16:04:12Z",
"pageFetchState": "SUCCESSFUL",
"googleCanonical": "https://tengrx.com/guides/nextjs-seo-audit/",
"userCanonical": "https://tengrx.com/guides/nextjs-canonical-url/",
"sitemap": ["https://tengrx.com/sitemap.xml"]
}
}
}
How Searchbop respects Google’s URL Inspection API quotas automatically
Google enforces a strict quota on the URL Inspection API (2,000 inspection requests per day and 600 per minute per Search Console property). Instead of re-querying every URL on every scan and exhausting your quota, Searchbop stores each URL’s inspection verdict and `checkedAt` timestamp in your local SQLite database. During an audit run, it prioritizes uninspected URLs and URLs whose cached verdict is older than 24 hours (inspecting up to 100 URLs per run with controlled concurrency).
Official API documentation & boundaries
What the URL Inspection API can and cannot do: The official Google Search Console URL Inspection API is read-only: it reports Google’s current indexed state for a URL, but does not offer an API endpoint to trigger “Request Indexing” in bulk (for regular web pages, Google uses XML sitemaps and natural crawling for discovery). Reference: Search Console API Usage Limits.
Research context & search queries covered on this page Audit · P0
Cluster & intent: audit & indexing (Commercial / Informational · P0/P1) · Target queries: website indexing checker, Google Search Console analysis tool, AI SEO audit, SEO audit tool, technical SEO audit software.
Related benchmark prompts: P01, P11, and P12.