Recommendations for webmaster friendly hosting services
Stay organized with collections
Save and categorize content based on your preferences.
Monday, September 14, 2009
Most of the recommendations we've made in the past are for individual webmasters running their
own websites. We thought we'd offer up some best practices for websites that allow users to
create their own websites or host users' data, like
Blogger or
Google Sites.
Make sure your users can verify their website in website management suites such as
Google's Webmaster Tools.
Webmaster Tools provides your users with detailed reports about their website's visibility in
Google. Before we can grant your users access, we need to verify that they own their
particular websites.
Verifying ownership of a site in Webmaster Tools
can be done using a custom
HTML file,
a
meta tag,
or seamless integration in your system via
Google Services for Websites.
Other website management suites such as
Yahoo! Site Explorer
and
Bing Webmaster Tools
may use similar verification methods; we recommend making sure your users can access each of
these suites.
Choose a unique directory or hostname for each user.
Webmaster Tools verifies websites based on a single URL, but assumes that users should be
able to see data for all URLs 'beneath' this URL in the site URL hierarchy. See our article on
verifying subdomains and subdirectories
for more information. Beyond Webmaster Tools, many automated systems on the web—such as
search engines or aggregators—expect websites to be structured in this way, and by doing
so you'll be making it easier for those systems to find and organize your content.
Set useful and descriptive page titles.
Let users set their own titles, or automatically set the pages on your users' websites to be
descriptive of the content on that page. For example, all of the user page titles should not
be "Blogger: Create your free blog". Similarly, if a user's website has more than one page
with different content, they should not all have the same title: "User XYZ's home page".
Allow the addition of tags to a page.
Certain meta tags are reasonably useful for search engines and users may want to control
them. These include tags with the name attribute of
robots,
description,
googlebot,
slurp,
msnbot. Click on the specific name attributes to learn more about what these tags
do.
Allow your users to use third-party analytics packages such as Google Analytics. Google Analytics is an
enterprise-class analytics software that can run on a website by just adding a snippet of
JavaScript to the page. If you don't want to allow users to add arbitrary JavaScript for
security reasons, the Google Analytics code only changes by one simple ID. If your let your
users tell you their Google Analytics ID, you can set up the rest for them. Users get more
value out of your service if they can understand their traffic better. For example, see
Weebly's support page
on adding Google Analytics. We recommend considering similar methods you can use for enabling
access to other third-party applications.
Help your users move around.
Tastes change. Someone on your service might want to change their account name or
even move to another site altogether. Help them by
allowing them to access their own data
and by letting them tell search engines when they move part or all of their site via the use
of 301 redirect destinations.
Similarly, if users want to remove a page/site instead of moving it, please return a
404 HTTP response code
so that search engines will know that the page/site is no longer around. This allows users to
use the
urgent URL removal tool
(if necessary), and makes sure that these pages drop out of search results as soon as possible.
Help search engines find the good content from your users.
Search engines continue to crawl more and more of the web. Help our crawlers find the best
content across your site. Allow us to crawl users' content, including media like user-uploaded
images. Help us find users' content using
XML Sitemaps. Help us to steer
clear of duplicate versions of the same content so we can find more of the good stuff your
users are creating by creating only one URL for each piece of content when possible, and by
specifying your canonical URLs
when not. If you're hosting blogs, create RSS feeds that we can discover in
Google Blog Search. If
your site is down or showing errors, please return 5xx response codes. This helps
us avoid indexing lots of "We'll be right back" pages by letting crawlers know that the content
is temporarily unavailable.
Can you think of any other best practices that you would recommend for sites that host users'
data or pages?
Posted by Greg Grothaus, Staff Software Engineer, Search Quality Team
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],[],[[["\u003cp\u003eUser-created websites should be easily verifiable in webmaster tools like Google Search Console, Yahoo! Site Explorer, and Bing Webmaster Tools for site management and analysis.\u003c/p\u003e\n"],["\u003cp\u003eEach user should have a unique directory or hostname for their website for better organization and indexing by search engines and other automated systems.\u003c/p\u003e\n"],["\u003cp\u003eAllow users to set descriptive page titles for each page on their website, or automatically generate them to enhance content understanding.\u003c/p\u003e\n"],["\u003cp\u003eEnable users to add relevant meta tags (robots, description, googlebot, slurp, msnbot) to their pages for better search engine optimization and control.\u003c/p\u003e\n"],["\u003cp\u003eEmpower users with insights into their website traffic by allowing the integration of third-party analytics packages like Google Analytics.\u003c/p\u003e\n"],["\u003cp\u003eFacilitate user control over their content by enabling data access, 301 redirects for site moves, and proper 404 error handling for removed content.\u003c/p\u003e\n"],["\u003cp\u003eEnhance search engine discovery of user-generated content through crawling permissions, XML Sitemaps, canonical URL specification, RSS feeds for blogs, and appropriate 5xx error codes for site downtime.\u003c/p\u003e\n"]]],["Websites hosting user-generated content should enable users to verify their sites in webmaster tools and provide unique directories/hostnames for each user. Implement descriptive page titles, allow `meta` tag additions, and support third-party analytics integration. Facilitate user data access, site/page moves via `301` redirects, and deletions via `404` errors. Aid search engine crawlers with XML Sitemaps, RSS feeds, canonical URLs, and proper `5xx` error codes during site downtime.\n"],null,["# Recommendations for webmaster friendly hosting services\n\n| It's been a while since we published this blog post. Some of the information may be outdated (for example, some images may be missing, and some links may not work anymore).\n\nMonday, September 14, 2009\n\n\nMost of the recommendations we've made in the past are for individual webmasters running their\nown websites. We thought we'd offer up some best practices for websites that allow users to\ncreate their own websites or host users' data, like\n[Blogger](https://blogger.com/) or\n[Google Sites](https://sites.google.com/).\n\n- **Make sure your users can verify their website in website management suites such as\n Google's Webmaster Tools.** \n Webmaster Tools provides your users with detailed reports about their website's visibility in Google. Before we can grant your users access, we need to verify that they own their particular websites. [Verifying ownership of a site in Webmaster Tools](https://support.google.com/webmasters/answer/34592) can be done using a custom [HTML file](https://www.google.com/support/webmasters/bin/answer.py?answer=35658), a [`meta` tag](https://www.google.com/support/webmasters/bin/answer.py?answer=35659), or seamless integration in your system via [Google Services for Websites](/search/blog/2009/08/new-tools-for-google-services-for). Other website management suites such as [Yahoo! Site Explorer](https://siteexplorer.search.yahoo.com/) and [Bing Webmaster Tools](https://www.bing.com/toolbox/webmasters/) may use similar verification methods; we recommend making sure your users can access each of these suites.\n- **Choose a unique directory or hostname for each user.** \n Webmaster Tools verifies websites based on a single URL, but assumes that users should be able to see data for all URLs 'beneath' this URL in the site URL hierarchy. See our article on [verifying subdomains and subdirectories](https://www.google.com/support/webmasters/bin/answer.py?answer=35163) for more information. Beyond Webmaster Tools, many automated systems on the web---such as search engines or aggregators---expect websites to be structured in this way, and by doing so you'll be making it easier for those systems to find and organize your content.\n- **Set useful and descriptive page titles.** \n Let users set their own titles, or automatically set the pages on your users' websites to be descriptive of the content on that page. For example, all of the user page titles should not be \"Blogger: Create your free blog\". Similarly, if a user's website has more than one page with different content, they should not all have the same title: \"User XYZ's home page\".\n- **Allow the addition of tags to a page.** \n Certain `meta` tags are reasonably useful for search engines and users may want to control them. These include tags with the `name` attribute of [`robots`](/search/blog/2007/03/using-robots-meta-tag), [`description`](/search/blog/2007/09/improve-snippets-with-meta-description), [`googlebot`](https://www.google.com/support/webmasters/bin/answer.py?answer=35264), [`slurp`](https://help.yahoo.com/l/au/yahoo7/search/indexing/indexing-11.html), `msnbot`. Click on the specific name attributes to learn more about what these tags do.\n- **Allow your users to use third-party analytics packages such as Google Analytics.** \n [Google Analytics](https://analytics.google.com/) is an enterprise-class analytics software that can run on a website by just adding a snippet of JavaScript to the page. If you don't want to allow users to add arbitrary JavaScript for security reasons, the Google Analytics code only changes by one simple ID. If your let your users tell you their Google Analytics ID, you can set up the rest for them. Users get more value out of your service if they can understand their traffic better. For example, see [Weebly's support page](https://www.weebly.com/support/index.php?pg=kb.page&id=23) on adding Google Analytics. We recommend considering similar methods you can use for enabling access to other third-party applications.\n- **Help your users move around.** \n Tastes change. Someone on your service might want to change their account name or even move to another site altogether. Help them by [allowing them to access their own data](https://www.mattcutts.com/blog/not-trapping-users-data-good/) and by letting them tell search engines when they move part or all of their site via the use of [`301` redirect destinations](/search/docs/crawling-indexing/301-redirects). Similarly, if users want to remove a page/site instead of moving it, please return a [`404` HTTP response code](/search/docs/crawling-indexing/http-network-errors#4xx-client-errors) so that search engines will know that the page/site is no longer around. This allows users to use the [urgent URL removal tool](https://www.google.com/support/webmasters/bin/answer.py?answer=92865) (if necessary), and makes sure that these pages drop out of search results as soon as possible.\n- **Help search engines find the good content from your users.** \n Search engines continue to crawl more and more of the web. Help our crawlers find the best content across your site. Allow us to crawl users' content, including media like user-uploaded images. Help us find users' content using [XML Sitemaps](https://www.sitemaps.org/). Help us to steer clear of duplicate versions of the same content so we can find more of the good stuff your users are creating by creating only one URL for each piece of content when possible, and by [specifying your canonical URLs](/search/blog/2009/02/specify-your-canonical) when not. If you're hosting blogs, create RSS feeds that we can discover in [Google Blog Search](https://blogsearch.google.com/). If your site is down or showing errors, please return `5xx` response codes. This helps us avoid indexing lots of \"We'll be right back\" pages by letting crawlers know that the content is temporarily unavailable.\n\n\nCan you think of any other best practices that you would recommend for sites that host users'\ndata or pages?\n\nPosted by [Greg Grothaus](https://gregable.com/), Staff Software Engineer, Search Quality Team"]]
