This page shows you how to add the structured data that search operators depend on.
Web pages are often filled with free form text, which is easy for humans to read but more difficult for computers to understand. Some web pages have information with greater structure that is easy to read, such as a page date embedded in the URL or title of the page, or machine-readable fields embedded in the HTML code. Google extracts a variety of structured data from web pages. This page describes the structured data types Google extracts that are available for use in Custom Snippets and Structured Search.
- Providing Data to Custom Search
- Providing Data to Rich Snippets
- Viewing Extracted Structured Data
When you are reading a webpage that sells a DVD, you can quickly figure out what the title is, what reviewers thought of the film, and how they rated it. But a computer cannot do the same things, because it doesn't understand how the information is structured.
For example, if the page has content about the DVD—along with recommendations for other items, ads from other stores, and comments from customers—then the page might have different prices for various things, not just for the DVD that is being sold. You can easily figure out the price for the DVD while dismissing the other prices, but the computer can't. Some sophisticated programs might find the prices in the webpage, but they cannot determine the rules for finding just the price of the DVD.
Structured data formats are rules that standardize the structure and content of the webpage. They are markup that you apply to text snippets so that computers can process their meaning or semantics. The markup does not change the formatting of your website, it just makes the metadata and text enclosed within the XHTML tags more meaningful to computers.
Custom Search recognizes the following formats:
- PageMaps: invisible blocks of XML that add metadata to pages.
- Microformats: tags used to mark up visible page content along predefined types.
- RDFa: an alternate standard for marking up visible page content along arbitrary types.
- Microdata: a new HTML5 standard for marking up visible page content.
<meta>tags: standard HTML tags, a subset of which are parsed by Google.
- Page Date: features on a page indicating its date, which Google attempts to parse
You can use one or a combination of formats that you prefer.
Note that unlike Custom Search, Google Search does not use PageMaps or
<meta> tags when generating rich snippets. Google Search does consider
information such as microformats, microdata, RDFa, and the page date
when it is generating snippet, but it has its own algorithm and policies
for determining what information gets shown to users. So while structured
data you add to your pages can be presented on Custom Search, it might not
be displayed in Google Search results.
The following includes an idealized snippet of plain HTML from a review site:
<div> <div> <h1>Pizza My Heart</h1> </div> <span>88%</span> like it <a href="#reviews">See all 12 reviews</a> <span>Under $10 per entree</span> <div>
The following snippet shows the previous HTML code extended with a format called microformats:
<div class="hreview-aggregate"> <div class="vcard item"> <h1 class="fn">Pizza My Heart</h1> </div> <span class="rating average">88%</span> like it <a href="#reviews">See all <span class="count">12</span> reviews</a> <span class="pricerange">Under $10 per entree</span> <div>
The Rich Snippets Testing Tool shows the information Google Search extracts from this page:
hreview-aggregate item hcard fn = Pizza My Heart rating average (normalized to 5.0 scale) = 4.5 average = 88% pricerange = Under $10 per entree count = 12
Custom Search uses a subset of the information available for Google Search; this subset is shown at the bottom of the testing tool page:
review (source = MICROFORMAT) ratingstars = 4.5 ratingcount = 12 pricerange = Under $10 per entree
By incorporating standard structured data formats into your webpages, you not only make the data available to Custom Search, but also for any service or tool that supports the same standard. Apply structured data to the most important information in the webpage, so you can present them directly in the results. For example, if you have a website selling Android devices, include structured data about the ratings, prices, availability, and whatnot. When your users search for the Android devices, they can see the ratings, prices, and availability at a glance.
So computers can now understand the types of data in the webpage. Now what? Computers can also start doing the menial task of finding and combining information in different webpages. This frees users from totally boring tasks, such as sifting through multiple pages to find items that they want. Search engines, such as Custom Search, can process the structured data in your webpages and display it in useful, more meaningful ways, such as custom snippets and structured search.
Providing Data to Custom Search
Google supports several kinds of data which are used primarily by
Custom Search: Pagemaps, a subset of
<meta> tags, and approximate page dates.
PageMaps is a structured data format that provides Google with information about the data on a page. It enables website creators to embed data and notes in webpages. Although the structured data is not visible to your users or to Google Web Search, Custom Search recognizes it when indexing your webpages and returns it directly in XML results or in JSON format in the Custom Search element.
You can explicitly add PageMaps to a page, or submit PageMaps using a Sitemap.
Google will also use other information on a page, such as rich snippets markup or
meta tag data, to create a PageMap.
Unlike the other structured data formats described below, PageMaps does
not require you to follow standard properties or terms, or even refer
to an existing vocabulary, schema, or template. You can just create
custom attribute values that make sense for your website. Unlike the structured
data attributes of microformats, microdata and RDFa, which are added around
user-visible content in the body of the HTML, PageMaps metadata is included in
head section of the HTML page. This method supports arbitrary
data which may be needed by your application but which you might not want to
display to users. (If you don't want PageMap information returned in your XML,
you can keep it private using an AccessKey.)
Once you create a PageMap, you can submit it to Google using any of the following methods:
- Add PageMap data directly to your HTML page. Google will discover the PageMap information when we crawl your site.
- Add PageMap data to a Sitemap, and submit that Sitemap for on-demand indexing. This is a good option if you don't want PageMap exposed in the HTML source code of your page.
- Submit PageMap data using the Custom Search Control API.
PageMap tag definitions
The following table outlines the requirements for adding PageMap data to a Sitemap.
||Yes||Encloses all PageMap information for the relevant URL.|
||Yes||Encloses all information about a single element (for example, an action).|
||Yes||Each DataObject contains one or more attributes.|
PageMaps are XML blocks and therefore must be formatted correctly;
in particular, the
Attribute tags in the XML are case sensitive, as are the
Add PageMap data directly to your HTML page
Here's an example of PageMap data for a webpage about badminton:
<html> <head> ... <!-- <PageMap> <DataObject type="document"> <Attribute name="title">The Biomechanics of a Badminton Smash</Attribute> <Attribute name="author">Avelino T. Lim</Attribute> <Attribute name="description">The smash is the most explosive and aggressive stroke in Badminton. Elite athletes can generate shuttlecock velocities of up to 370 km/h. To perform the stroke, one must understand the biomechanics involved, from the body positioning to the wrist flexion. </Attribute> <Attribute name="page_count">25</Attribute> <Attribute name="rating">4.5</Attribute> <Attribute name="last_update">05/05/2009</Attribute> </DataObject> <DataObject type="thumbnail"> <Attribute name="src" value="http://www.example.com/papers/sic.png" /> <Attribute name="width" value="627" /> <Attribute name="height" value="167" /> </DataObject> </PageMap> --> </head> ... </html>
Add PageMap data to a Sitemap
If you don't want to include PageMap data in the HTML of your pages, you can add PageMap data to a Sitemap and submit that Sitemap for on-demand indexing using the Custom Search Control Panel.
Here's an example of a Sitemap that includes PageMap information for two URLs: http://www.example.com/foo and http://www.example.com/bar.
<?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <url> <loc>http://www.example.com/foo</loc> <PageMap xmlns="http://www.google.com/schemas/sitemap-pagemap/1.0"> <DataObject type="document" id="hibachi"> <Attribute name="name">Dragon</Attribute> <Attribute name="review">3.5</Attribute> </DataObject> </PageMap> </url> <url> <loc>http://www.example.com/bar</loc> <PageMap xmlns="http://www.google.com/schemas/sitemap-pagemap/1.0"> <DataObject type="document" id="biggreenegg"> <Attribute name="name">Ribs</Attribute> <Attribute name="review">4.0</Attribute> </DataObject> </PageMap> </url> </urlset>
Submit PageMap data using the Custom Search Control API
To submit PageMap data using the Custom
Search Control API, send an HTTP
POST message, using the
text/xml content type, to:
Include PageMap data in the message body, like this:
<OnDemandIndex> <Pages> <Page url="http://www.example,com/monkeys/"> <pagemap> <DataObject type="document"> <Attribute name="title">Monkeys</Attribute> <Attribute name="review">4.0/5.0</Attribute> </DataObject> <DataObject type="stats"> <Attribute name="installs">1000</Attribute> <Attribute name="comments">100</Attribute> </DataObject> </pagemap> </Page> <Page url="http://www.example.com/parrots"> <pagemap> <DataObject type="document"> <Attribute name="title">Parrots</Attribute> <Attribute name="review">4.5/5.0</Attribute> </DataObject> <DataObject type="stats"> <Attribute name="installs">2000</Attribute> <Attribute name="comments">200</Attribute> </DataObject> </pagemap> </Page> </Pages> </OnDemandIndex>
In some cases, you may not want custom attributes returned in your search
engine's query results XML, because those are publicly visible by default. In
this case, you can create a private PageMap by adding an AccessKey to
the DataObject you want to protect, and sending the PageMap directly to Google
using the on-demand indexing API. Only web
searches with a matching
AccessKey parameter will get that
DataObject in results.
Here's an example of an AccessKey in use:
<Page url="http://www.example.com/monkeys/"> <pagemap> <DataObject type="stats"> <AccessKey>myprivate12345</AccessKey> <Attribute name="installs">1000</Attribute> <Attribute name="comments">100</Attribute> </DataObject> </pagemap> </page>
An AccessKey can consist of no more than 30 alphanumeric characters.
To retrieve protected data, specify the AccessKey in the
pgmpk parameter. For example, if your AccessKey is
myprivate12345, your query URL might look like this:
To restrict results to protected data, update your search
URL to append the AccessKey value to the
operator, like this:
To sort by protected data, update your search URL to append
the AccessKey value to the
Parsing PageMap data
If you are getting results back via XML, then the custom attributes are returned in the results within the PageMap tag, as shown below. You can parse the DataObjects within the PageMap tag and provide customized presentation of the relevant attributes. If you are using the Custom Search element, then the custom attributes are returned in the richSnippet property of each result for use in data templates, as described at Rich Snippet result properties.
<r n="1"> <u> http://www.xyz.com/business/vending_machine.html </u> ... <t> In Italy, a Vending Machine Even Makes the <b>Pizza</b> </t> ... <s>The European vending machine industry has annual sales of about #33 billion, much of it coming from factories and offices.</s> ... <PageMap> <DataObject type="image"> <Attribute name="image_src" value="http://www.nytimes.com/images/2009/03/14/business/14vend.751.jpg"/> </DataObject> <DataObject type="publication"> <Attribute name="author" value="John Tagliabue"/> <Attribute name="date" value="March 14, 2009"/> <Attribute name="category" value="Business/World Business"/> </DataObject> </PageMap> ... </r>
While PageMaps allow you to precisely specify the data you want for
each page, sometimes you have a large amount of content which you do
not want to annotate. Google extracts selected content from
tags of the form
content="VALUE">. We do not support variants of the
META tag, such as the use of
property instead of
While we explicitly exclude common
tags that are usually inserted programmatically by web authoring tools,
keywords, rarer tags specific to your site will be
extracted and put into a special data object
metatags, which can be used with all of Custom
Search's structured data features. For example, a
<meta> tag of the form:
<meta name="pubdate" content="20100101">
creates a PageMap DataObject which is returned in XML results like this:
<r n="1"> ... <PageMap> <DataObject type="metatags"> <Attribute name="pubdate" value="20100101"/> </DataObject> </PageMap> ... </r>
The data in this automatically created PageMap can be used anywhere you can use data from a PageMap explicitly included in your page's content. For instance, it can be used with structured search operators like Sort by Attribute:
or with the Custom Search element:
... <gcse:search sort_by="metatags-pubdate:d:s"></gcse:search> ...
<meta> tags excluded by Google include:
Google attempts to include all other
<meta> tags, with the caveat that
punctuation, special characters and embedded spaces in the
<meta> tags may not be parsed correctly. Custom Search
explicitly supports periods and dashes in
<meta> tag names.
Custom Search does not explicitly support other special characters
<meta> tag names, but some special characters
may be accepted correctly if they are
Custom Search will convert up to 50
<meta> tags to PageMaps, as long
as the total text size of all processed properties does not exceed 1MB, with no
individual property exceeding 1024 characters.
Using Page Dates
In addition to metadata which you explicitly specify on a page,
Google also estimates a page date based on features of the page such
as dates in the title and URL. Custom Search allows you to use this
date to sort, bias and range restrict results by using a special metadata
date. This estimated date can be used in all operators
that use the
&sort= URL parameter, including
Sort by Attribute,
Bias by Attribute,
Restrict to Range.
Note: The page date is not added to the PageMap, so it is not returned in XML results, cannot be used in the Custom Search element, and cannot be used with the Filter by Attribute feature.
The following examples show the use of the page date with these operators:
|If you want to...||Send this URL...||To learn more see...|
|Sort results by date in descending order||
||Sort by Attribute|
|Bias results strongly towards newer dates||
||Bias by Attribute|
|Bias results weakly towards older dates||
||Bias by Attribute|
|Return results from January 1 to February 1 of 2010 (inclusive)||
||Restrict to Range|
Google's estimate of the right date for a page is based on features such as the byline date of news articles or an explicitly specified date in the title of the document. If a page has poorly specified or inconsistent dates Google's estimate of the page date may not make sense, and your custom search engine may return results ordered in a way you do not expect.
A site may provide date information implicitly, relying on Google's estimated page date feature to detect dates embedded in the page URL, title or other features, or explicitly, by supplying a date in a structured data format. In either case, effective use of dates requires formatting the dates correctly.
For Custom Search's Sort by Attribute, Bias by Attribute, Restrict to Range features, Google attempts to parse dates using both conventional date formatting and formal standards such as ISO 8601 and IETF RFC 850. The following complete date formats are accepted:
|Date Format||Example Date|
|Month DD YYYY||December 31 2009|
|DD Month YYYY||31 December 2009|
Google will attempt to parse variants of these date formats, such
the more ambiguous the date, the less likely that Google will parse
it correctly. For example, the date
extremely ambiguous and it is unlikely Google will assign to it
the interpretation you want. For best results, use a complete
date format with a fully specified year.
Google also extracts a variety of structured data from Microformats, RDFa
and Microdata to be used in
Rich Snippets, extended presentations of standard Google search results.
A subset of this data is available for use in Custom Search's
structured data operators—typically, the same data used in Rich Snippets.
For example, if you have marked up your pages with the Microformat
hrecipe standard, you could sort on the number of rating
stars of the recipe with an operator like
Google is continually extending the data it extracts and how much of this
data is available for use in Custom Search; to see what data we currently
extract, you can use the
Rich Snippets Preview Tool in Webmaster tools.
is a specification for representing commonly published
items such as reviews, people, products, and businesses. Generally,
microformats consist of
<div> elements and a class property, along with a
brief and descriptive property name (such as
rating, which represent the date an item was reviewed
and its rating, respectively).
The following includes a snippet of plain HTML code.
<p><strong>Kevin Grendelzilla</strong></p> <p>Technical writer at Google</p> <p>555 Search Parkway</p> <p>Googlelandia, CA 94043</p>
The following snippet shows the previous HTML code extended with microformats:
<div class="vcard"> <p><strong class="fn">Kevin Grendelzilla</strong></p> <p><span class="title">Technical writer</span> at <span class="org">Google</span></p> <p><span class="adr"> <span class="street-address">555 Search Parkway</span> <span class="locality">Googlelandia</span>, <span class="region">CA</span> <span class="postcode">94043</span> </span></p> </div>
Google extracts a subset of this data, normalized and reorganized to correspond to how it would be displayed in Rich Snippets. This subset would be returned in XML results like this:
<r n="1"> ... <PageMap> <DataObject type="person"> <Attribute name="location" value="Googlelandia"/> <Attribute name="role" value="Technical Writer"/> </DataObject> </PageMap> ... </r>
To see what Google extracts for a page, use the Rich Snippets Testing Tool in Google's Webmaster Tools site. The data Google extracts from pages is continually being extended, so check back periodically to see if the data you want has been made available. In the meantime, if you need custom data that does not correspond to a defined microformat, you can use PageMaps.
Using Resource Description Framework in Attributes (RDFa)
Resource Description Framework in attributes (RDFa) is more flexible than microformats. Microformats specify both a syntax for including structured data into HTML documents and set of microformat classes each with its own specific vocabulary of allowed attributes. RDFa, on the other hand, specifies only a syntax and allows you to use existing vocabularies of attributes or create your own. It even lets you combine multiple vocabularies freely. If the existing vocabularies do not meet your needs, you can define your own standards and vocabularies by creating new fields.
The following includes a snippet of plain HTML code.
<div> <h3>5 Centimeters Per Second</h3> <h4>Makoto Shinkai</h4> ... </div>
The following snippet shows the previous HTML code extended with RDFa:
<div> <h3 property="dc:title">5 Centimeters Per Second</h3> <h4 property="dc:maker">Makoto Shinkai</h4> ... </div>
HTML5, the latest revision of the language web pages are written in,
defines a format called
that incorporates the ideas of RDFa and Microformats directly into the
HTML standard itself. Microdata uses simple attributes in HTML tags
div) to assign brief and
descriptive names to items and properties.
Like RDFa and Microformats, Microdata's attributes help you specify that your content describes information of specific types, like reviews, people, information or events. For example, an person can have the properties name, nickname, url, title and affiliation. The following is an example of a short HTML block showing this basic contact information for Bob Smith:
<div> My name is Bob Smith but people call me Smithy. Here is my home page: <a href="http://www.example.com">www.example.com</a> I live in Albuquerque, NM and work as an engineer at ACME Corp. </div>
The following is the same HTML marked up with microdata. Note that in this example we use a property 'nickname' that is not yet officially part of schema.org. Custom Search is a good way to explore possible schema.org extensions locally before proposing them to the wider community.
<div itemscope itemtype="http://schema.org/Person"> My name is <span itemprop="name">Bob Smith</span> but people call me <span itemprop="nickname">Smithy</span>. Here is my home page: <a href="http://www.example.com" itemprop="url">www.example.com</a> I live in Albuquerque, NM and work as an <span itemprop="title">engineer</span> at <span itemprop="affiliation">ACME Corp</span>. </div>
The first line of this example includes a HTML
div tag with
itemscope attribute that indicates that
contains a microdata item. The
itemtype="http://schema.org/Person" attribute on
the same tage tells us this is a person. Each property of the person item
is identified with the
itemprop attribute; for example,
itemprop="name" on the
span tag describes
the person's name. Note that you are not limited to
itemprop="url" tag is attached
a (anchor) tag.
Viewing Extracted Structured Data
After you have tagged your webpages with structured data, you can use the Rich Snippets Testing Tool to view the structured data that can be extracted from the webpage. The tool provides two views: the first view shows the structured data that Google Search can extract from the page, while the second view shows what Custom Search can extract from the page.
If you haven't tagged any of your webpages but would like to see what extracted structured data might look like, you can enter the URL of other websites. Popular sites that have review information or list of contacts are more likely to have structured data. If you see result snippets on Google search that looks similar to Figure 1, you can conclude that the webpage has structured data.
Figure 1: Result snippet with rating, price range, and review.
Once you have found a page with structured data, you can view that page's source to see the structured data that site has implmented, or view that page in the Rich Snippets Testing Tool to see what data is extracted for Google Search rich snippets and Custom Search structured search. For example, consider the following snippet of HTML with structured data about a person implemented as microformats:
<div class="vcard"> <h1 class="fn"> <span class="given-name">Godzilla</span> <span class="family-name">Gigantis</span> </h1> <span class="title">Senior Giant Monster</span>, <span class="adr"> <span class="locality">Tokyo</span> </span> <div>
From a page with this markup, Google extracts the following data for use in rich snippets:
hcard fn = Godzilla Gigantis n family-name = Gigantis given-name = Godzilla adr locality = Tokyo title = Senior Giant Monster
Custom Search extracts the following subset of that data for use in structured search:
person (source = MICROFORMAT) location = Tokyo
Thus, this tool allows you to view not only the Rich Snippets markup
recognized for Google Search, but also the additional customized
markup that we support in Custom Search. You can immediately see how
your web page would be processed during indexing, and what metadata
attributes would be returned in PageMaps in your Custom Search results.
If there are any errors in your markup, you can fix them right away.
Remember, you need to add the
&view=cse parameter to the
URL or click the checkbox to review the additional metadata
extracted by Custom Search.
Exploring Other Features
Structured data can be used in several Custom Search features including the following:
- If you want to learn more about using structured data in snippets, see Customizing Your Result Snippets.
- If you want to learn more about changing the order of your results, see Structured Search.
If you want to write client applications that dynamically create custom search engines using HTTP request methods, see Programmatically Creating Custom Search Engines.