You published. Six hours later, nothing. Not even a crawl. This checklist walks you through the five most common reasons new pages stay out of the index, from rogue noindex tags to orphaned URLs. No fluff. No theory.
Quick calculator. Put in the expected monthly value of a page or link batch and the natural waiting time.
You hit publish. The URL is live. But Google has not even sniffed it. The problem is rarely mysterious. It is almost always one of five things: a hidden noindex tag, a missing or stale sitemap, zero internal links, a crawl budget limit, or a content quality floor that Google considers too low to index.
In practice, when you open Search Console and see 'Discovered - currently not indexed', the first thing you should check is whether the page is actually in the sitemap. A common situation we see in audits: an SEO team submits a sitemap containing 500 URLs, but the CMS automatically adds a noindex meta tag to new pages during a staging-to-production migration. The sitemap tells Google the page exists; the tag tells Google to ignore it. The result? A contradiction that wastes your crawl budget.
This checklist cuts through that noise. You will diagnose, fix, and verify in under 20 minutes.
| Blocker | Where It Hides | Immediate Fix | Failure Mode If Ignored |
|---|---|---|---|
| Rogue noindex tag Often inherited from a template or plugin default | <meta name='robots' content='noindex'> in head or HTTP X-Robots-Tag header | Remove the tag or set index in your CMS. Use Google's URL Inspection Tool to test live. | Google finds the page via sitemap, obeys the tag, and never indexes it. You get 'Excluded by noindex' in coverage report. |
| Orphan page No internal links point to it | Not reachable from any sitemap or nav link. Only exists as a direct URL. | Add at least one contextual link from an indexed page. Prioritize pages with existing crawl traffic. | Google may never discover the URL. Even if submitted in sitemap, it may sit in 'Crawled - currently not indexed' for weeks. |
| Sitemap mismatch URL is not in a submitted sitemap, or sitemap is stale | Check sitemap index file and individual sitemaps in Search Console. | Add the URL to the correct sitemap, resubmit in Search Console, and request indexing via the Inspection Tool. | No sitemap entry = lower priority for discovery. Google might still find it via links, but it will take much longer. |
| Crawl budget cap Googlebot is busy crawling other pages on your site | Log analysis or Search Console 'Crawl stats' report. Look at crawl rate and total crawled pages per day. | Block low-value pages (facets, parameter-heavy URLs) in robots.txt. Increase crawl rate if your server can handle it. | Important new pages may never get crawled. When they do, it is often weeks later. Large sites (50k+ pages) are most affected. |
Open the page in a browser. View page source. Search for 'noindex'. If present, remove it and test again.
Go to Search Console > URL Inspection. Enter the URL. Check the 'Coverage' status. If it says 'URL is not on Google', request indexing.
Check your XML sitemap. Is the URL present? If not, regenerate the sitemap. If yes, check the <lastmod> tag - it should reflect the publish date.
Use a crawler (Screaming Frog or Sitebulb) to find all internal links to the page. If count is 0, add links from at least two different indexed pages.
Check robots.txt. Ensure the page path is not disallowed. Also check for any X-Robots-Tag in HTTP response headers.
Review server logs for Googlebot hits on the URL. If no hits after 48 hours, consider a low crawl budget or a disallow rule.
Test the page content for thinness. If it has fewer than 300 words of unique content, expand it. Google sometimes skips very short pages.
View page source or use browser dev tools. If <meta name='robots' content='noindex'> is present, remove it and resubmit.
Is the URL in a submitted sitemap? If not, add it. If yes, check lastmod date matches publish date.
Use a crawler. Count internal links pointing to the new page. If 0, add at least 2 contextual links from indexed pages.
In Search Console, go to Crawl stats. If your site has a low crawl rate, block low-value pages and increase crawl rate.
In URL Inspection Tool, click 'Request indexing'. Wait 24-48 hours. Re-check status.
The scenario: You published a 1,200-word guide on 'Drip-feed indexing strategies' 30 days ago. It has zero organic traffic. Search Console shows 'Discovered - currently not indexed'.
Step 1: Open the URL in the Inspection Tool. The live test shows noindex is not present. Good.
Step 2: Check the sitemap. The URL is missing from the blog sitemap because your WordPress SEO plugin was set to exclude posts older than 7 days from the sitemap (a common setting for news sites). You have 30 posts in the sitemap. This post should be there. Fix: Change the sitemap setting to include all posts, or set a custom <lastmod> date to today.
Step 3: Internal links. Using Screaming Frog, you find exactly 0 internal links to this post. The only way to reach it is through the blog archive. Fix: Add a contextual link from the homepage (which has a crawl rate of 200 pages/day) and from a related post that is already indexed.
Step 4: Resubmit the sitemap and request indexing. Within 18 hours, the URL changes to 'Submitted and indexed'.
Blocked by robots.txt but sitemap submitted. This is a contradiction. Google will still crawl the URL if it is in the sitemap, but it will obey the disallow and not index the content. The fix: remove the disallow rule for the directory or page.
Google discovers the page but sees no unique value. This happens with thin affiliate pages or pages that are 90% near-duplicate of an existing page on your site. Even if you remove noindex, Google may choose not to index it. Add original analysis, data, or a unique angle.
Soft 404. Your page loads fine for humans, but Google sees a 404 or 410 status code. This often occurs when a CMS serves a 404 template but returns a 200 status. Use the Inspection Tool's live test to check the HTTP response code.
JavaScript rendering issues. If your content is loaded via JavaScript and the version served to Googlebot is empty, the page may be seen as blank. Test with 'Fetch as Google' (in the Inspection Tool) to see the rendered HTML.
| Option | What happens | Verdict |
|---|---|---|
| <strong>Sitemap submission</strong><br>Tells Google the URL exists and its last modification time. Essential for discovery. Does not guarantee crawl. | <strong>Internal linking</strong><br>Tells Google the URL is important. Contextual links pass relevance signals. The strongest signal for indexing priority. | Both are required. Sitemaps for discovery, internal links for priority. |
| <strong>Request Indexing (in Search Console)</strong><br>A manual ping. Works for individual URLs. Can be used up to 10 times per URL per day. No guarantee of indexing. | <strong>Drip-feed indexing</strong><br>Gradually publishing and linking new pages to control link velocity. Useful for large-scale launches. Reduces risk of algorithmic penalties. | Use manual requests for urgent single pages. Use drip-feed for batch launches. |
| <strong>Pinging services (e.g., Bing Webmaster Tools, Pingomatic)</strong><br>Not used by Google. Only affects Bing and smaller search engines. Low impact. | <strong>Social sharing</strong><br>Can accelerate discovery if the page gets organic shares. Google uses social signals as a crawl trigger but not as a ranking signal. | Social sharing is a bonus, not a primary tactic. |
If you manage a site with more than 5,000 pages, manual checks do not scale. Use the Google Search Console API to programmatically check the index status of a list of URLs. You can pull the coverage report and filter for 'Discovered - currently not indexed' or 'Crawled - currently not indexed'.
In practice, when you run an API query for a client with 50,000 product pages, you often find that 12% of new products are not indexed because the sitemap regeneration cron job fails silently. The API uncovers this in seconds. Pair it with a log analyzer to see which URLs Googlebot actually hit.
For a deeper workflow on managing link velocity and preventing algorithmic penalties, read the drip-feed indexing guide on Medium. It covers the operational side of scheduling links to avoid triggering spam filters.
Most likely an internal linking issue. A sitemap tells Google the URL exists, but without at least one contextual link from an already-indexed page, Google may deprioritize the crawl. Add two links from other content pages and resubmit the sitemap.
The API does not directly show meta tags. Instead, use the URL Inspection API to get the index status. If the response shows 'noIndex' as true, the page has a noindex directive. You can batch-query up to 2000 URLs per day with the API.
First, ensure the page is in the sitemap and has at least one internal link from a category page that Google crawls daily. Then use the URL Inspection Tool to request indexing. On large sites, also check crawl budget: block low-value faceted URLs in robots.txt to free up budget.
No, but if you are linking back to your own new page from a guest post, that link can help discovery. The issue is usually on your own site. Check if your new page has internal links and is in the sitemap. Guest post backlinks are a secondary signal.
Google found the URL (usually via sitemap or a link), but chose not to index it yet. Common causes: low crawl budget, thin content, or the page is considered low quality. Fix by adding more unique content, improving internal links, and resubmitting.
Use the Search Console API with a custom script. Pull the 'indexStatus' for each URL from the coverage report. Filter for 'notIndexed' or 'indexed'. If you are not a developer, use a tool like Sitebulb that exports index status from Search Console data.
Yes. 'Crawled' means Googlebot visited the page but chose not to index it. 'Discovered' means Google found the URL (via sitemap or link) but has not crawled it yet. Both are fixable, but 'Crawled' often indicates a content quality issue, while 'Discovered' is usually a crawl budget or link issue.
Three errors: (1) the URL returns a 404 or 301, (2) the <lastmod> date is older than the actual publish date, causing Google to ignore it, and (3) the sitemap index file is not properly referencing the child sitemaps. Validate your sitemap in Search Console to see errors.
Parse your server logs to find all requests from Googlebot. Check if your new page URL appears in the logs. If not, Google has not attempted to crawl it. If it appears multiple times but the coverage report still shows 'not indexed', the page likely has a noindex tag or is serving a soft 404.
Indirectly, yes. If your server is slow, Googlebot may reduce its crawl rate, meaning fewer pages are crawled per day. New pages on slow sites often sit in 'Discovered' for weeks. Fix by improving server response time (TTFB under 200 ms) and increasing crawl rate in Search Console.