We onboarded a dental practice last year that had invested $8,000 in a new website. Custom design. Great content. Proper schema markup. They launched it, waited two months, and saw zero organic traffic. Not low traffic β zero.
When we looked at Google Search Console, the reason was immediately obvious: none of their pages were indexed. Not one. The site existed on the internet, but as far as Google was concerned, it didn't.
The culprit was a single checkbox in their WordPress settings that said "Discourage search engines from indexing this site." The developer had turned it on during the staging phase and forgotten to uncheck it when the site went live. Two months of invisible existence because of one checkbox.
That's an extreme example, but indexing problems are far more common than most people realize. And if your pages aren't indexed, nothing else you do in SEO matters. You can have the best content on the internet β if Google hasn't added it to the index, no one will ever find it through search.
Indexing is the process by which Google adds a web page to its searchable database. Think of Google's index like a massive library catalog. When Google "indexes" your page, it's filing a card for that page in the catalog. When someone searches for something, Google checks the catalog and decides which cards β which pages β are the most relevant results.
If your page isn't in the catalog, it can't show up in search results. Period. It doesn't matter how good the page is.
Indexing is actually the second step in a three-step process. Here's how the whole thing works:
The key thing most people miss: crawling and indexing are not the same thing. Google can crawl your page β meaning its bots visited and read it β without actually indexing it. If Google decides the page is low quality, duplicate, or not useful enough to include in the index, it will crawl it and then walk away.
There are two quick ways to check. The first takes about three seconds:
Type a URL above and we'll open a site: search in Google for that specific page. If Google shows a result, the page is indexed. If it shows nothing, it isn't.
The more thorough method is Google Search Console. The "Pages" report in Search Console tells you exactly how many pages are indexed, how many are not, and β critically β why they're not indexed. Every site we manage is connected to Search Console, and it's the first place we look when diagnosing traffic problems.
Go to Search Console β Pages (under Indexing in the sidebar). You'll see two numbers: indexed pages and not-indexed pages. Click "Not indexed" to see a breakdown of reasons. Common statuses include "Discovered β currently not indexed," "Crawled β currently not indexed," and "Excluded by noindex tag." Each one tells you something different about why Google isn't including those pages.
After auditing hundreds of sites, these are the indexing problems we encounter most frequently, ranked roughly by how often we see them:
A noindex meta tag tells Google explicitly not to index a page. Sometimes this is intentional β you don't want your internal search results pages or staging pages in Google's index. But more often than we'd like to admit, it's accidental.
We've seen WordPress plugins add noindex to entire post types. We've seen developers leave noindex tags from the staging environment on production pages. We've seen Yoast settings that noindex all category pages by default β including category pages that were meant to rank.
<head> section of your page's HTML for <meta name="robots" content="noindex">. In WordPress, check your SEO plugin settings (Yoast β Search Appearance, or Rank Math β Titles & Meta). Also check Settings β Reading for the "Discourage search engines" checkbox.Your robots.txt file tells search engines which parts of your site they're allowed to crawl. If a page (or an entire directory) is blocked by robots.txt, Google can't crawl it β and if Google can't crawl it, it can't index it.
The tricky part: sometimes developers block directories that contain important resources. We've seen robots.txt files that block the /wp-content/uploads/ directory, preventing Google from reading images. We've seen rules that block entire subdirectories of the site without anyone realizing it.
yoursite.com/robots.txt. Use Google's robots.txt tester in Search Console to verify that your important pages aren't being blocked. Make sure your XML sitemap URL is included in the robots.txt file.Google doesn't have to index every page it crawls. If it determines that a page doesn't provide enough unique value to justify inclusion in the index β because the content is too thin, too similar to another page, or simply not useful β it'll skip it.
This shows up in Search Console as "Crawled β currently not indexed." It means Google saw the page, evaluated it, and decided it wasn't worth including. This is Google's way of saying "we don't think this page is good enough."
If a page isn't linked to from anywhere else on your site, Google may never discover it β even if it's in your sitemap. These are called "orphan pages." They exist on your server but have no connection to the rest of your site's link structure.
We see this a lot with blog posts that were published but never linked to from category pages, related posts sections, or the main navigation. Google's crawler follows links. If there are no links to follow, the page is effectively invisible.
Google allocates a "crawl budget" to each site β a limit on how many pages it will crawl within a given timeframe. For most small-to-medium sites, this isn't a problem. But for larger sites with thousands of pages, low-quality pages, or slow server response times, Google might run out of budget before reaching your important pages.
This is especially relevant for sites that generate hundreds of thin pages automatically β tag archive pages, parameter-based filter pages, paginated results. Those pages consume crawl budget that could be spent on your high-value content.
An XML sitemap is a file that lists all the pages you want Google to index. It's not strictly required β Google can discover pages through links β but it significantly speeds up the discovery and indexing process, especially for new pages.
We regularly find sites with no sitemap at all, sitemaps that return 404 errors, sitemaps that haven't been updated in years, or sitemaps that list URLs returning errors. Each of these scenarios slows down indexing.
Sometimes the answer is just patience. New pages can take anywhere from a few hours to several weeks to be indexed, depending on your site's crawl frequency and authority. A page on the New York Times gets indexed in minutes. A page on a brand-new website with no authority might take 2β4 weeks.
If a page has been live for over a month and still isn't indexed, that's a problem worth investigating. If it's been three days, give it time.
Not every page on your site should be indexed. Some pages actively dilute your site's quality signal if they're in Google's index. Here's our general rule for what to index and what to keep out:
The general principle: if a page provides unique value to a searcher, index it. If it exists only for internal site functionality or would add thin content to Google's impression of your site, noindex it.
For our clients in financial services, law, and healthcare, indexing issues carry additional weight because the pages themselves are more valuable. A single well-indexed service page for a personal injury attorney can generate $50,000β$200,000 in annual case revenue. A dermatologist's procedure page ranking for "Botox near me" can drive $300,000+ in treatment bookings per year.
When one of those pages isn't indexed, the cost isn't theoretical. It's real revenue that's going to competitors whose pages are indexed and ranking.
The audit that pays for itself: We've found unindexed high-value pages on nearly every new client site we audit. Last quarter, we discovered that a law firm's highest-potential service page β targeting a keyword with $180 CPCs β had been accidentally noindexed for eight months. Fixing that single issue was worth more than a year of SEO retainer fees.
Indexing is the most fundamental requirement of SEO. It's not glamorous. Nobody writes case studies about getting a page indexed. But if your pages aren't in Google's index, everything else β content, links, schema, speed optimization β is completely irrelevant.
The good news is that indexing problems are usually straightforward to identify and fix. Connect your site to Google Search Console. Check the Pages report. Look for pages that are "not indexed" and read the reasons Google gives you. Most issues come down to accidental noindex tags, robots.txt blocks, thin content, or orphan pages with no internal links.
If you've never checked your indexing status, do it today. You might be surprised by how many of your pages Google is quietly ignoring.
Our free SEO audit includes a complete indexing analysis β how many pages Google has indexed, which pages are missing, why they're excluded, and exactly what to fix. It's one of the first things we check, because nothing else matters until this is right.
Our free audit checks your indexing status, identifies missing pages, and tells you exactly why Google is excluding them β plus the fix for each one.