DSPL Cookbook

This document contains a cookbook of useful DSPL and canonical concept "recipes". These provide step-by-step directions for a variety of tasks that go above and beyond the basic language features described in the Tutorial and Developer Guide. The content below assumes knowledge of the previous documents, so be sure to read through them before beginning your "cooking".

Each recipe begins with a goal followed by a specific list of steps to follow. Many also include DSPL snippets or links to external examples. If you have any feedback about a recipe or would like to suggest others, please post a message in the DSPL Forum.

Basic Recipes

Specifying display names

Goal

Associate a "display name" with each instance of a (dimension) concept. These names will show up in the Public Data Explorer UI instead of the concept instance IDs, which are generally shorter and harder for users to understand.

Steps

  1. Make your concept extend entity:entity.
  2. Add a name column to the associated concept definition table.
  3. Populate the latter property with the name of each instance.

Notes

  • There is no need to explicitly define a name property in your concept metadata; this definition is automatically included when you extend entity:entity.
  • You can also add description and info_url columns to provide instance-level descriptions and URLs, respectively.

Creating concept hierarchies

Goal

Create a hierarchy of (dimension) concepts. These will show up in the Public Data Explorer UI as a tree, allowing users to understand the groupings and navigate between them.

Steps

  1. Define a "parent" concept (e.g., region).
  2. Define a "child" concept (e.g., subregion).
  3. Add a property in the child that references the parent and includes the isParent attribute:
    <property concept="..." isParent="true"/>
    
    where the dots are replaced by the parent ID.
  4. In the definition table for the child, add a column that references the parent.
  5. For each child instance, populate the latter with a valid instance of the parent concept.

Notes

  • See the country / state relationship in the tutorial dataset for an example of this.
  • Each parent ID referenced from the child must be in the parent's definition table. Otherwise, the importer will produce an error.
  • The parent and child can be the same concept (i.e., a self-referencing hierarchy). See the business concept in the US Retail Sales Dataset for an example of this.

Categorizing concept instances

Goal

Create categories for concept instances that will show up as options in the Public Data Explorer color pickers.

Steps

  1. Define a "category" concept (e.g., income_level).
  2. Define a "child" concept (e.g., country).
  3. Add a property in the child that references the category concept and includes a name element:
    <property concept="..."/>
      <info>
        <name><value>"..."</value></name>
      </info>
    </property>
    
    where the first set of dots is replaced by the category concept ID and the second set is replaced by the desired display name for your property (e.g., "Income Level").
  4. In the definition table for the child, add a column that references the category concept.
  5. For each child instance, populate the latter with a valid instance of the category concept.

Notes

  • A concept can have multiple properties that serve as categories; in this case, the user will see all of them as options in the color pickers.
  • One of the categories can also be a parent for the purposes of organizing the instances hierarchically in the UI. See the previous recipe for details.

Specifying the sort order for concept instances

Goal

Specify the sort order for instances of a concept. This order is used when listing the instances in the left-nav of the Public Data Explorer visualization pages.

Steps

  1. Make your concept extend entity:entity or one of its children (e.g., geo:location).
  2. Add an entity_order attribute to your concept metadata.
  3. Set the value of the previous to one of ALPHA or TABLE; the former will cause the instances to be listed in order of their display names, whereas the latter will keep the instances in the same order as they appear in the concept definition table.

Notes

  • ALPHA is the current default, so if you'd like to use this ordering, the above steps are optional.
  • See the entity:entity documentation for an example.

Fetching data from remote sources at import time

Goal

Fetch table data CSVs from remote sources instead of bundling these with your dataset.

Steps

  1. Put one or more of your data table CSVs in a site or service that allows public access via HTTP, HTTPS, or FTP; examples of the former include Google Cloud Storage and Amazon S3.
  2. Provide the URL as a file name, e.g.:
    <table id="..."/>
      ...
      <data>
        <file format="csv" encoding="utf-8">http://storage.googleapis.com/dspl/countries.csv</file>
      </data>
    </table>
    
  3. Zip and upload your dataset normally, omitting any files referenced via URLs. The latter will be fetched automatically at import time.

Notes

  • The data are only fetched at import time. The Public Data Explorer does not yet support dynamic fetching of data at the time of visualization.
  • To refetch remote data, just upload a new version of your dataset.
  • Your data URLs must be publicly accessible. The Public Data Explorer cannot fetch data from password-protected sources.

Fetching data from Fusion Tables at import time

Goal

Fetch table data CSVs from queries on one or more Google Fusion Tables.

Steps

  1. Create a Fusion Table, and make it publicly accessible via the sharing settings in the Fusion Tables UI.
  2. Provide a Fusion Tables query in the data section of your table metadata using the following syntax:
    <data>
      <external type="fusiontables">
        <query>
          ...
        </query>
      </external>
    </data>
    where ... is replaced by your query. See the Fusion Tables API documentation for more information on the supported query syntax.
  3. Zip and upload your dataset normally.

Notes

  • The Fusion Tables system is only queried at import time. The Public Data Explorer does not yet support dynamic querying of data at the time of visualization.
  • To rerun the queries and update your data, just upload a new version of your dataset.
  • All referenced Fusion Tables must be publicly accessible.
  • The query strings do not have to be URL-encoded.

Time Recipes

Quarterly data

Goal

Visualize data that is specified at quarterly intervals.

Steps

  1. Convert all of the quarters in your slice data to months (e.g., the first month of the quarter).
  2. Treat this dimension as a time:month in your slice XML.

Weekly data

Goal

Visualize data that is specified at weekly intervals.

Steps

  1. Convert all of the weeks in your slice data to days (e.g., the first day of the week).
  2. Treat this dimension as a time:day in your slice XML.

Geo Recipes

Using country or state canonical concepts

Goal

Use the canonical country or state concepts, importing all of the lat/long values, etc., so that these don't need to defined within the dataset.

Steps

  1. Make sure that all of the country and/or US state references in your slice data CSVs are valid IDs from the canonical country and state definition tables.
  2. Add geo:country and/or geo_us:state as dimensions in the corresponding slices.
  3. Make sure that the column names for these match the names of the previous concepts (not including the imported dataset name, e.g., country) or, if not, include mapDimension statements in your slice definition. Example:
    <slice id="...">
      ...
      <dimension concept="geo:country"/>
      ...
      <mapDimension concept="geo:country" toColumn="my_country"/>
    </slice>
    

Using only a subset of the canonical countries or states

Goal

Use a subset of a canonical geo concept so that only this subset (and not the entire list, e.g. all the countries in the world) appears in the Public Data Explorer UI.

Steps

  1. Define a local concept that extends the superset concept. Example:
    <concept id="my_country_subset" extends="geo:country">
    ...
    </concept>
    
  2. In the associated concept definition CSV, only include the IDs of the superset concept that you want to use. Example:
    my_country_subset
    FR  
    MX
    US
    

Notes

  • Each instance of your subset concept must be a valid instance of the superset concept; you cannot "expand" the concept with new instances.

Defining your own geographic concepts

Goal

Create your own geographic concepts that are explorable and mappable, just like the canonical, Google-defined ones.

Steps

  1. Create a concept that extends geo:location.
  2. In the associated concept definition table, include the columns name, latitude, and longitude.
  3. Populate the latter properties for each instance of your concept; the latitude and longitude values are floats with the convention that N and E are positive, and W and S are negative.

Notes

  • There is no need to explicitly include the name, latitude, and longitude properties in your concept definition as these are automatically included when extending geo:location.
  • See the definition of state in the tutorial dataset for an example.

Metric and Unit Recipes

Using percents

Goal

Define a (metric) concept that is a percentage of something else.

Steps

  1. Make your concept extend quantity:ratio.
  2. Set the is_percentage attribute to true.
    <attribute id="is_percentage">
      <value>true</value>
    </attribute>
    
  3. Optionally, set the percentage_of attribute based on what this concept is a percentage of. Example:
    <attribute id="percentage_of">
      <value>labor force</value>
    </attribute>
    

Using currency units

Goal

Define a (metric) concept that has currency units (e.g., US dollar).

Steps

  1. Make your metric extend quantity:quantity or one of its children, e.g. quantity:amount.
  2. Add a unit attribute that references unit:currency and set the value of this to an ID from the associated currency definition table. Example:
    <concept id="...">
      <attribute concept="unit:currency">
        <value>EUR</value>
      </attribute>
    </concept>
    

Creating a custom unit

Goal

Create a custom unit for one or more of your metrics.

Steps

  1. Create a concept that extends unit:unit.
  2. Add the columns unit_text and symbol to the associated definition table. Optionally, if you'd like to adjust the symbol position, you can also add a symbol_position column.
  3. Fill in the values for the latter properties for each instance of your unit. Example CSV for "area" units:
    my_unit,unit_text,symbol,symbol_position
    SQKM,square kilometers,km²,END
    SQML,square miles,miles²,END
    
  4. To use your newly created unit in a (metric) concept, make sure the latter extends quantity:quantity or one of its children. Then, add an attribute that references your unit concept. Example:
    <concept id="..." extends="quantity:amount">
      ...
      <attribute concept="my_unit">
        <value>SQKM</value>
      </attribute>
      ...
    </concept>
    

Notes

  • If your unit concept has only a single instance, you can omit the ID column from its definition table as well as the value tags in the metrics that use it.
  • See the definition of the canonical currency concept for a complete example.