Mueller Explains The Limits Of Index Coverage Report

Google SEO reports are a vital part of any website optimisation strategy. However, several SEOs have been noticing a bug in their Index Coverage Report. Responding to the issue, Google said that the Index Coverage Report does not provide up-to-the-minute coverage data. For those looking for the most up-to-date confirmation of whether a URL is indexed or not, the search engine company recommends using the URL Inspection Tool for Google SEO.

Apparently, the “bug” caused the Index Coverage Report to report that a URL had been crawled but not included in Google’s index. This isn’t actually a bug, however; it only happens due to the Index Coverage Report’s limitations.

As explained by Google in several tweets, the Index Coverage data is refreshed at a different (and slower) rate than the URL Inspection. The results in URL Inspection are more up-to-date, and if they conflict with those in the Index Coverage report, they should be considered more authoritative.

Google said that the Index Coverage data should indicate the actual status of a page in a few days when the condition changes. Nevertheless, they thanked the SEO community for their comments. The search engine company said that they would try to minimise this discrepancy so that their Google SEO reports and tools would consistently be up-to-date.

John Mueller Answers A Query About Index Coverage Report

On 8 October 2021, Google’s John Mueller responded to a query about this issue. This was before people understood that the Index Coverage Report simply had a gap between the expectations of data freshness and the fact that data is refreshed at a slower rate.

In July 2021, a user said that they noticed URLs submitted via Google Search Console reporting the error of submitted but not indexed, even though there was no noindex tag on the pages. Google would visit the website again after that, crawl it, and index it as usual.

They claimed that the problem is that they receive 300 errors/no index on their first crawl, but only five are crawled on subsequent attempts before re-crawling so many more. So, assuming they are noindexed and that things can’t render or locate the page, they’re sent to Page Not Found, which has a noindex tag. The person who asked the question speculated that there might be a memory issue.

Mueller said it is difficult to say without seeing the pages. He said he’d double-check if this was an issue now and then recheck it a few weeks later to see whether or not it was still a problem. He would also check if the problem intermittently happens because it does not matter if the problem’s already been solved.

The person who asked the question disputed this, claiming that the problem still occurs and is an ongoing issue. Mueller stated that if the event continues to occur, he would try to discover what might be causing it. He speculates that there may be something wrong with the rendering.

And it’s possible that after testing the page in Search Console, nine times out of ten, it performs well. But it’s conceivable that when something goes wrong, it will take users to the error page.

Mueller said that it’s also best if they try to figure out whether there are too many requests to render the page or if there’s something complicated with the JavaScript – maybe it takes too long sometimes, and other times it works well. He told the person to narrow things down from that point of view.

Mueller then described how Google’s crawling and rendering processes work from their side. He refers to a “Chrome-type” browser, which may be a nod to Google’s headless Chrome bot, a Chrome browser with no user interface.

He claimed that Google first crawls the HTML page and then runs it through a Chrome-like browser. And, to do so, they rely on all of the resources listed there. If one goes to the Developer Console in Chrome and examines the network section, they will see a waterfall representation of everything that is downloaded to create the page. If there are a lot of items to load, it’s possible that things will time out, and they’ll eventually encounter that error.

Reducing the number of resource requests for JavaScript and CSS files is a good idea, as well as attempting to combine or minimise them and reduce the size of images.

Mueller’s advice is connected to Rendering SEO, which was mentioned by Google’s Martin Splitt. The technical details of how a web page is downloaded and displayed in a browser are optimised for faster and better performance.

Some Crawl Errors Are Server Related

Mueller’s response was not entirely relevant in this case since the problem was about the freshness expectations rather than indexing.

However, his advice is still useful for the many times that a server-related problem causes resource serving timeouts, which block the correct rendering of a website.

This may happen at night or in the early hours of the day when a site is overrun by bots and is slowed down.

A poor-quality website without optimised resources, particularly one that is hosted on a shared server, can suffer from significant slowdowns where the server returns 500 error response codes. Nginx, Apache, or PHP configuration errors at the server level or a failing hard drive can all lead to a website not displaying requested pages to Google or visitors.

Some problems may appear when different programs are changed to suboptimal settings, requiring troubleshooting to discover issues. Fortunately, some software like Plesk have diagnostic and repair tools that can help correct these issues if they occur.

The problem this time was that Google had not set the proper expectation for the Index Coverage Report. But next time, it might be a server or rendering problem.

Need Help Managing Your GSC? Hire Position1SEO Now!

Google Search Console is a great way to track the indexing of your site and any errors you may be experiencing. If it’s not reporting up-to-date data, we can help with that! Position1SEO has been working in the search engine optimisation industry for years, so we know how Google works.

We have experience running SEO campaigns on all types of sites, from e-commerce stores to company websites and everything in between. We’re the UK’s top SEO firm because our no-stone-unturned approach to Google SEO ensures that your Google rankings are always as high as they can be.

With our high-quality SEO services, you can expect to see a significant boost in your website’s traffic. Not only that, but we can also help you retain your existing userbase and even convert them into paying customers!

So, what are you waiting for? Let us partner with you today!