Smartphone crawl errors in Webmaster Tools

Wednesday, December 04, 2013

Some smartphone-optimized websites are misconfigured in that they don't show searchers the information they were seeking. For example, smartphone users are shown an error page or get redirected to an irrelevant page, but desktop users are shown the content they wanted. Some of these problems, detected by Googlebot as crawl errors, significantly hurt your website's user experience and are the basis of some of our recently-announced ranking changes for smartphone search results.

Starting today, you can use the expanded Crawl Errors feature in Webmaster Tools to help identify pages on your sites that show these types of problems. We're introducing a new Smartphone errors tab where we share pages we've identified with errors only found with Googlebot for smartphones.

Smartphone Crawl Errors view in Webmaster Tools

Some of the errors we share include:

  • Server errors: A server error is when Googlebot got an HTTP error status code when it crawled the page.
  • "Not found" and soft 404 errors: A page can show a "not found" message to Googlebot, either by returning an HTTP 404 status code or when the page is detected as a soft 404 page.
  • Faulty redirects: A faulty redirect is a smartphone-specific error that occurs when a desktop page redirects smartphone users to a page that is not relevant to their query. A typical example is when all pages on the desktop site redirect smartphone users to the home page of the smartphone-optimized site.
  • Blocked URLs: A blocked URL is when the site's robots.txt explicitly disallows crawling by Googlebot for smartphones. Typically, such smartphone-specific robots.txt disallow rules are erroneous. You should investigate your server configuration if you see blocked URLs reported in Webmaster Tools.

Fixing any issues shown in Webmaster Tools can make your site better for users and help our algorithms better index your content. You can learn more about how to build smartphone websites and how to fix common errors. As always, please ask in our forums if you have any questions.