Creating Datasets and Importing Data

AI-generated Key Takeaways

• Datasets can be created with a display name and usage type specified through a POST request.

• Datasets can be imported from Google Cloud Storage (GCS) by providing the GCS file path and file format in a POST request to the import endpoint.

• Datasets can also be imported from local files using a multipart upload, including metadata and the raw data file, sent to the import endpoint.

• The responses for successful dataset creation and import operations include the dataset's unique name or identifier.

Create a Dataset

Send a request to create a dataset.

curl -X POST -d '{
    "display_name": "DISPLAY-NAME", 
    "usage": "USAGE_DATA_DRIVEN_STYLING"
  }' \
  -H 'X-Goog-User-Project: PROJECT_NUMBER' \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  https://mapsplatformdatasets.googleapis.com/v1alpha/projects/PROJECT_NUMBER/datasets

Sample Request

curl -X POST -d '{
    "display_name": "John Test Dataset", 
    "usage": "USAGE_DATA_DRIVEN_STYLING"
  }' \
  -H 'X-Goog-User-Project: PROJECT_NUMBER' \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  https://mapsplatformdatasets.googleapis.com/v1alpha/projects/802317041956/datasets
  

Sample Response

{
  "name":
  "projects/802317041956/datasets/f57074a0-a8b6-403e-9df1-e8a9e4f9fc46",
  "displayName": "John Test Dataset",
  "usage": [
    "USAGE_DATA_DRIVEN_STYLING"
  ],
  "createTime": "2022-08-15T17:50:00.189682Z",
  "updateTime": "2022-08-15T17:50:00.189682Z" 
}

Import a Dataset

via GCS

Send a request to import a dataset from a file in GCS.

curl -X POST  -d '{
    "gcs_source":{
      "input_uri": "GCS_FILE_PATH",
      "file_format": "FILE_FORMAT"
    }
  }' \
  -H 'X-Goog-User-Project: PROJECT_NUMBER' \
  -H "content-type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  https://mapsplatformdatasets.googleapis.com/v1alpha/projects/PROJECT_NUMBER/datasets/DATASET_ID:import

Sample Request

curl -X POST -d '{
    "gcs_source":{
      "input_uri":"gs://john-test-bucket/UnsafeRoads.kml",
      "file_format": "FILE_FORMAT_GEOJSON"
    }
  }' \
  -H 'X-Goog-User-Project: PROJECT_NUMBER' \
  -H "content-type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  https://mapsplatformdatasets.googleapis.com/v1alpha/projects/802317041956/datasets/f57074a0-a8b6-403e-9df1-e8a9e4f9fc46:import

Sample Response

{
  "name": "projects/802317041956/datasets/f57074a0-a8b6-403e-9df1-e8a9e4f9fc46@66c7e658-fa0a-4023-99b0-72bf7515cca6"
}

via Local File

Send a request to import a dataset from a local file.

curl -X POST \
  -H 'X-Goog-User-Project: PROJECT_NUMBER' \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Goog-Upload-Protocol: multipart" \
  -F "metadata=@METADATA_FILE" \
  -F "rawdata=@RAWDATA_FILE" \
  https://mapsplatformdatasets.googleapis.com/upload/v1alpha/projects/PROJECT_NUMBER/datasets/DATASET_ID:import

Sample METADATA FILE

{"local_file_source": {"file_format": "FILE_FORMAT_GEOJSON"}}

Sample RAWDATA FILE

{
  "type": "FeatureCollection",
  "name": "Bridges",
  "features": [
    {
      "type": "Feature",
      "properties": { "OBJECTID": 1, "UNITDESC_BRG": "E VIRGINIA DOCK"},
      "geometry": {
        "type": "LineString",
        "coordinates":
            [ [ -122.329201929090928, 47.616910448708538 ], [ -122.329206483407461, 47.616976719821004 ],
              [ -122.32921802149356, 47.617042137515213 ], [ -122.329236413912909, 47.617105967923777 ],
              [ -122.329261454336034, 47.617167494985758 ], [ -122.329292861855023, 47.617226028479571 ],
              [ -122.329330284134699, 47.617280911766009 ] ] }
    },
    {
      "type": "Feature",
      "properties": { "OBJECTID": 2, "UNITDESC_BRG": "E-N RAMP"},
      "geometry": {
          "type": "LineString",
          "coordinates":
            [ [ -122.313478277471646, 47.549939305158262 ], [ -122.313463749124196, 47.549869552219029 ],
              [ -122.313455987690375, 47.549798726337066 ], [ -122.31345506493939, 47.549727482432473 ],
              [ -122.313460989403836, 47.549656479290768 ] ] }
    }]
}

Sample Response

{
  "name": "projects/802317041956/datasets/f57074a0-a8b6-403e-9df1-e8a9e4f9fc46@66c7e658-fa0a-4023-99b0-72bf7515cca6"
}