01 / NOT INDEXED VS. NOT RANKING

Step 1: Separate “not indexed at all” from “indexed, but not ranking for your query”

Before changing a single line of code, determine whether Google knows your website exists:

If Search Console says **URL is not on Google**, you have a discovery, crawling, or indexing block (Steps 2–6 below). If Search Console says **URL is on Google**, your technical plumbing is working, and your challenge is search relevance, internal linking, or content depth (Steps 7–10).

02 / THE 5 HARD TECHNICAL BLOCKERS

Steps 2–6: The 5 technical mistakes that keep a website completely out of Google

When an entire site or newly launched section fails to appear in Google, one of these five configurations is almost always responsible:

Step & Blocker Where It Hides Exact Fix
Step 2: Leftover noindex Directive HTML <meta name="robots" content="noindex">, CMS “Discourage search engines” checkbox, or hosting X-Robots-Tag: noindex header. Remove the `noindex` directive so production pages serve `index, follow`, then request re-indexing in GSC.
Step 3: robots.txt Blocking Crawlers https://yourdomain.com/robots.txt contains User-agent: * followed by Disallow: / from development. Update `robots.txt` to `Allow: /` for `Googlebot`, `Bingbot`, `OAI-SearchBot`, and `*`, and add your `Sitemap:` URL.
Step 4: Broken Canonical URL <link rel="canonical"> points to `http://localhost:3000`, a `*.vercel.app` / `*.pages.dev` preview domain, or the homepage `/` on every child page. Configure `metadataBase` or your CMS canonical generator so every page outputs an absolute, self-referencing production canonical URL.
Step 5: Wrong GSC Property Variant You verified `http://yourdomain.com` in Search Console, while your live site redirects to `https://www.yourdomain.com`. Add a **Domain property (`sc-domain:yourdomain.com`)** via DNS TXT verification so Search Console tracks all protocols and subdomains together.
Step 6: Client-Side Empty HTML Shell Single-Page Application (React/Vite/Vue) renders <div id="root"></div> with client-only data fetching that fails or times out during rendering. Pre-render or server-render (SSR/SSG) critical page titles, headings, body text, and internal <a href> links in the initial HTML response.
03 / WHY INDEXED PAGES STILL DON’T SHOW UP

Steps 7–10: Why your pages are indexed (or crawled) but don’t appear when people search

If your technical checks pass and your sitemap is submitted, check these four editorial and structural signals:

04 / ONE-MINUTE TERMINAL DIAGNOSTIC

How to check your live HTTP headers and HTML tags from the terminal

Browser extensions often inspect the hydrated DOM rather than the raw HTTP response Googlebot receives. Run these two commands in your terminal (or run an automated audit in Searchbop) to see the exact truth:

TERMINAL DIAGNOSTIC COMMANDSChecking HTTP status, X-Robots-Tag, canonical, and meta robots
# 1. Check HTTP status code, redirect chain, and X-Robots-Tag header
curl -ILs https://yourdomain.com/ | grep -iE "^HTTP/|location:|x-robots-tag:"

# 2. Check rendered HTML <title>, meta robots, and canonical link tag
curl -s https://yourdomain.com/ | grep -iE "<title>|name=\"robots\"|rel=\"canonical\""

Official Google Search Central documentation

Reference documentation: See Google’s official guide Why is my page missing from Google Search? and the Crawling and Indexing FAQ. Remember that crawling and indexing can take anywhere from a few days to several weeks after publishing or fixing a technical blocker, and no tool can guarantee indexing or ranking.

Research context & search queries covered on this page Indexing · P1

Cluster & intent: indexing & audit (Informational · P1) · Target queries: why is my website not showing on Google, Google indexing troubleshooting, website indexing checker, sitemap submitted but not indexed.

Related benchmark prompts: P01, P11, and P12.