Mueller Explains Why GSC Discovers URLs But Doesn’t Index Them

Online businesses and site owners use Google Search Console for crawling and indexing in SEO. However, many of them are annoyed by the “discovered but not indexed” notification. Google’s John Mueller recently explained to the SEO community why Google sometimes chooses not to index a URL.

Many SEOs discussed the matter on Facebook and Twitter. They have received notifications about URLs being discovered but not getting indexed. Seeing such notifications may bring a lot of frustrations because it’s unpleasant to work on a piece of content only for Google to see it and then reject it.

One SEO specialist started a Twitter thread about this subject to explain how their newly-published post was discovered but not indexed by Google. The person said that one of their client sites had two new articles. More than ten days went by, and Google Search Console said the URL was discovered but currently not indexed.

The SEO submitted the URLs for a re-crawl; however, Google refused to index them. As a result, they assumed that maybe the URL was blocked after being discovered and not indexed. They then decided to remove the old URL and replace the content’s link with a new one. They also re-published the new web page with a new date.

After replacing the content’s link, the SEO was surprised to see that the content – now with a new URL – was indexed immediately in just a few hours. Moreover, they haven’t even submitted it to Google Search Console yet. As for the other articles that were left alone, the tool has completely ignored and never indexed them. Therefore, the SEO decided to move them to a new URL and give them a slightly new published date and see if the same thing happens.

Because of the little experiment, the SEO specialist concluded that there must have been an issue with the URL. They went on to explain that they fully know that Google indexes articles using the URL as the main “id”, which all signals are associated with. Therefore, it makes sense to replace the URL if it gets a “discovered but not indexed” notification on its first time.

Mueller replied to the thread, saying that there is nothing special about the “discovered but not indexed” notification. He said that it’s just a feature that tells SEOs that Google saw the content, but they might index it later or maybe never. Mueller then asked the SEO community if the feature was too confusing and whether Google should just remove it.

Mueller acknowledged that, in rare cases, Google might index a piece of content under a new URL when the website is “teetering on the edge of indexing”. This situation could mean many things, such as when there are issues with the overall site quality. So, even if an SEO submits content with a new URL, it can drop out again several days later, or some other links might; there will always be fluctuation. Therefore, SEO experts and site owners would need to convince Google that their content is worth indexing.

He then further explained that since Google didn’t have an understanding of the unindexed URL, they would have to look at the entire website to gain more insight into the content’s potential context within the rest of the web. They want to find out if the content is something that the Internet has been waiting for or if it’s just another red widget.

Other SEOs also joined in the thread, asking Mueller how to convince Google to index their content. Mueller simply told them to make their content “awesome”.

What “Awesomeness” Means

It’s already obvious that being amazing is a good idea; it doesn’t require explanation. It is, however, somewhat vague. Mueller could have been telling the SEO community to stop doing what everyone else is doing and instead do what they think is best for their SEO. Or he must have been encouraging everyone to create content that would have made them feel eager to read it when they were new to the topic.

SEOs must also let go of the idea that they should do things “ten times better”. Although it is motivational, “more” does not equate to “better”. Another frequent mistake is to follow the competitors by using the same keywords and synonyms. There is no secret in their keywords, and copying them only results in rewritten content.

Therefore, the SEO community need not be so obsessed with stealing ideas from what is currently ranking in the top ten. It makes sense to know what Google ranks, but one should stop rewriting what’s already ranking.

If a content writer or a publisher already knows the subject and are excellent at it, they should do their best instead of focusing on their competitors. Perhaps Google will identify a distinctive voice on their site; others may call it awesome, and it should have no trouble being indexed.

What Causes The “Discovered But Not Indexed” Notification

Larger websites with over 10,000 pages may get the “discovered but not indexed” notification due to many factors. For instance, Google may think the site is overloaded and be unable to crawl it.

Another factor is that some websites may have too much content than what Google is willing to crawl at the moment, so they think the article is not worth indexing. For instance, some sites have auto-generated and user-generated content and filtered product category pages.

Another reason is that the website has a poor internal link structure, so Google refuses to crawl the content. Sometimes, it is caused by poor content quality that does not satisfy the user intent.

Ensure That Your Pages Are Well-Optimised With Position1SEO

For smooth crawling and indexing in SEO, one should ensure that their web pages are well-optimised for Googlebot. Here at Position1SEO, we can help make that happen!

We are the UK’s top SEO agency, providing our clients with unique content that satisfies user intent and provides value to readers. We can also handle your Google Search Console account, helping you stay on top of any issues or updates from Google.

Our team follows a unique SEO process combined with white-hat tactics and a no-stone-unturned approach. This means we work in detail, and we create in-depth reports for each enquiry you make, giving you an idea of how successful your campaign has been so far, as well as what may be necessary to achieve your goals.

We are confident in our years of experience in the SEO industry, and we can guarantee that your website ranks above your competitors.

Contact us today for any queries about our SEO services!