Build the Backend with Vehicle Simulator

This tutorial shows you how to create the Node.js backend component of the Transport Tracker, using a vehicle location simulator. The simulated locations are useful during development and testing of your application, so that you can run the Transport Tracker without having to move Android devices around to represent your vehicles.

The backend receives updates of vehicle locations from the Firebase Realtime Database, and sets up the panel configurations for use by the Transport Tracker map.

The tutorial includes a link to the code repository, setup instructions, and a detailed walkthrough of the main parts of the code.

Get the code

Clone or download the Transport Tracker repo on GitHub.

Set up a Firebase Realtime Database

Skip this step if you have already set up a Firebase Realtime Database for your Transport Tracker.

The Transport Tracker uses a Firebase Realtime Database to communicate location updates between the various components of the server and front end applications. When the vehicle locator, or the simulator, stores the location updates in the Firebase Realtime Database, Firebase sends automatic updates to the backend, which in turn updates the front end display.

Set up your Firebase Realtime Database project:

  1. Go to the Firebase console and click Add project to create a project for your Transport Tracker.
  2. Enter a project name.
  3. Click Create Project.

Get a Google Maps API key

Skip this step if you have already obtained a Maps API key for your Transport Tracker.

Click the button below, which guides you through the process of registering a project in the Google Cloud Platform Console, activates the required Google Maps Platform and related services automatically, and generates a generic, unrestricted API key.

Get Started

Make a note of the value of your API key.

During development and testing, you can use a single, unrestricted Google API key for all the Google Maps Platform in your Transport Tracker. When you are ready to move your system into production, you must create separate API keys for each type of API, so that you can secure the keys by HTTP referrer or IP address. For help, see the pre-launch checklist.

Set up the backend and run the application

Follow these steps to set up the backend:

  1. Get your Firebase web app credentials:

    • Go to the Firebase console and open your Firebase project.
    • Click Add Firebase to your web app.
    • Go to the Service Accounts tab in your Firebase project's settings.
    • Click Generate New Private Key and confirm the action as prompted.
    • Firebase downloads an adminsdk JSON file containing your service account credentials. Store this file for later use.
  2. Optional: You can use Google Cloud services and the Google Cloud Shell to build and run your application:

    • Go to Google Cloud Platform console, sign in using your Google Account (Gmail or Google Apps) and create a new project.
    • Enter a name for your project, and take note of your allocated project ID.
    • Activate Google Cloud Shell from the Cloud platform console, by clicking the button at top right of the Cloud Shell toolbar.
    • The Cloud Shell window opens at the bottom of the Cloud console. This is where you can enter commands to interact with the shell. Refer to the Cloud Shell documentation for details.
  3. In Google Cloud Shell, or locally if you're not using Google Cloud, create a transport-tracker directory for your application, and copy across the backend directory from the cloned GitHub repo:

    mkdir transport-tracker
    cd transport-tracker
    cp <my-cloned-repo>/backend/ .
    cd backend
  4. Edit the file serviceAccountKey.json, and paste in your Firebase adminsdk service account credentials from the file you downloaded earlier. Hint: If you're using the Google Cloud Shell, you can use Cloud's code editor.

  5. Edit the tracker_configuration.jsonfile and add the following values:

    • mapsApiKey - your Maps API key. If you don't have one yet, follow the guide to getting an API key.
    • databaseURL - the address of your Firebase Realtime Database. You can find the URL in the Admin SDK configuration snippet on the Firebase Service Accounts tab.
    • simulation - a configuration setting that determines whether to use the vehicle simulator or real location data from the vehicle locator. While developing and testing, set this value to true.
  6. Run npm install to install your dependencies. This may take a few minutes.

  7. Run the application:

    npm run main
  8. Open your Firebase Realtime Database to see the results:

You should see data in your Firebase Realtime Database, something like this:

Transport Tracker Firebase Realtime Database

Understand the code

This guide focuses on the Google Maps Platform used in the Transport Tracker. Where relevant, the guide mentions other concepts and related code. You can get the full code from the GitHub repository linked at the top of the page.

It's helpful if you have a basic understanding of Node.js and npm.

Data flow

Below is a conceptual data flow diagram for the Transport Tracker, focusing on the backend modules.

Transport Tracker data flow

Node.js modules

These are the modules that make up the backend. Refer to the diagram to see how they fit together:

bus_simulator.js Defines the BusSimulator class, which simulates the positions of buses. It uses the time signal from HeartBeat to look up bus locations in the generated paths.json store.
gtfs.js Defines the GTFS class, which loads the GTFS (General Transit Feed Specification) data into an in-memory SQLite database, and provides functionality to query the GTFS timetable.
heart_beat.js Defines the HeartBeat class, which publishes time updates, either real or simulated, to the Firebase Realtime Database.
main.js The main logic for the Node.js application.
panel_changer.js Defines the PanelChanger class, which publishes the relevant part of the panels_config.json file. The front end uses this file to determine the geographical area it needs to display, along with the appropriate hotel markers for each panel.
road_snapper.js Defines the RoadSnapper class, which uses the Roads API to snap GPS signals to actual roads.
time_table.js Defines the TimeTable class, containing the core business logic of the Transport Tracker. In summary:
  • Query the GTFS database in response to time change notifications from the HeartBeat instance, via the time published in the Firebase Realtime Database.
  • Gather predicted travel times from the Directions API and cache the results.
  • Group the transformed data in accordance with the panel configuration for the map.

Generate fake vehicle paths for use in the simulator

While you're developing and testing, it's useful to have a simulated feed of vehicle locations, instead of needing live data from the vehicle locator. As a once-off step, you can prepare and store a set of vehicle paths for use by the Transport Tracker's simulation module. The GitHub repo includes a pre-generated set of paths in paths.json. Note: This is a very large file.

The GitHub repo also includes the generate_paths.js module, which you can use to generate the paths yourself:

generate_paths() calls the Directions API to plot a route from one location to another. It uses the Node.js Client for Google Maps Services to interact with the Directions API, and bluebird as a JavaScript promise library, along with asyncawait, to handle asynchronous communications.

Start the application from main.js

The starting point for the Transport Tracker backend is main.js.

The following excerpt from the main.js file sets up your credentials for the Firebase Admin SDK, which you obtained from Firebase earlier in this tutorial. For details, see the Firebase documentation.

const trackerConfig = require('./tracker_configuration.json');
const admin = require('firebase-admin');
const serviceAccount = require('./serviceAccountKey.json');

  credential: admin.credential.cert(serviceAccount),
  databaseURL: trackerConfig.databaseURL

The database references define the paths in your Firebase Realtime Database:

  • bus-locations: The locations of the vehicles on the map.
  • map: The bounds of the map, and the particular set of route cards that appear on the map at any one time.
  • panels: The timetable data (routes and estimated times of arrival) shown on the map.
  • promo: Short descriptions of the components of the Transport Tracker, shown in an information panel at bottom right of the map.
  • current-time: The time stamp that determines the timetable and related data shown on the map.
const busLocationsRef = admin.database().ref('bus-locations');
const mapRef = admin.database().ref('map');
const panelsRef = admin.database().ref('panels');
const promoRef = admin.database().ref('promo');
const timeRef = admin.database().ref('current-time');

The core logic of main.js determines whether to use the vehicle simulator or the real location data from the vehicle locator, based on the value of the simulation setting in the configuration file:

const gtfs = new GTFS();
new HeartBeat(timeRef, trackerConfig.simulation);
new TimeTable(timeRef, panelsRef, gtfs, panelConfig, googleMapsClient);
new PanelChanger(mapRef, panelConfig);
new PromoChanger(promoRef);
if (trackerConfig.simulation) {
  const {BusSimulator} = require('./bus_simulator.js');
  const generatedPaths = require('./paths.json');
  new BusSimulator(timeRef, gtfs, busLocationsRef, generatedPaths);
} else {
  // Exercise for the reader: integrate real bus location data

Load and parse the GTFS timetable data

The timetable information is available as a set of CSV files in GTFS (General Transit Feed Specification) format. (The sample data in the GitHub repository is from the Google I/O Bus Tracker.)

The gtfs.js module defines the GTFS class, which loads the timetable data into an in-memory SQLite database. The class offers a number of methods giving access to the data, so that you can figure out where a specific bus should be at a specific time. Examples of the methods are getTripsForCalendarDate() and getStopInfoForTrip().

Simulate the location of moving vehicles

While you're developing and testing, it's useful to have a simulated feed of vehicle locations, instead of needing live data from the vehicle locator. Based on the value of the simulation setting in the application configuration file, the core logic in main.js (described above) determines whether to use a vehicle simulator (bus_simulator.js) or the real vehicle location data.

The bus_simulator.js module creates fake vehicle locations and pushes them to the Firebase Realtime Database, based on:

  • The routes in the GTFS timetables.
  • A set of generated paths in generate_paths.js, created by generate_paths.js (see above).
  • A time signal from HeartBeat (see below).

Snap the GPS locations to actual roads

A series of latitude/longitude coordinates often doesn't give an accurate indication of a route that follows a road. This is true whether the coordinates originate as GPS signals from the vehicle locator or as estimated coordinates from the simulator.

You can call the Roads API to snap the geographical coordinates to the actual road traveled. road_snapper.js uses the Node.js Client for Google Maps Services to interact with the Roads API, and bluebird as a JavaScript promise library, along with asyncawait, to handle asynchronous communications.

Generate a stream of time updates

At the heart of the application is the HeartBeat class, which sends a stream of updates to the Firebase Realtime Database using either simulated time or real time, depending on whether you're using the vehicle simulator or live updates from the vehicle locator.

Publish the timetable

The TimeTable class subscribes to the time updates in the Firebase Realtime Database, generated by HeartBeat. At each time update, TimeTable does the following:

  • Query the GTFS database to find the next scheduled trips.
  • Call the Directions API to get predicted travel time for each route.
  • Get the grouping of routes per UI panel from panels_config.json.
  • Construct the UI panels for the map.

Swap the UI panels at regular intervals

The Transport Tracker map displays one of three geographical areas at any one time, along with the routes and markers applicable to that area. The markers indicate hotels and bus stops.

The panels_config.json file defines the three panels. Each panel defines the geographical bounds of the area that appears in that panel, and the routes and markers that are relevant for that geographical area.

The PanelChanger class rotates the panels, based on the configuration in panels_config.json.

Other notable files and packages

As well as the Node.js modules, the backend includes these files:

generate_paths.js Constructs a prediction of where the vehicles will be at given times.
gtfs directory, containing the following files:
The timetable information for the tracked vehicles. The sample data is based on the timetable for the Google I/O Bus Tracker. This tiemtable is stored in a set of CSV files that follow GTFS, the General Transit Feed Specification.
package.json The information required by npm to correctly download the development and runtime dependencies, and to run the resulting application.
panels_config.json The configuration for the panel layout in the Transport Tracker map. Defines the three panels of the display, each with a group of routes.
paths.json Predicted vehicle locations and times. The generate_paths.js script creates this file. Note: Don't open this file in the editor, as it's very large.
serviceAccountKey.json The credentials for your Firebase service account.
tracker_configuration.json Various configuration values, including the Google Maps API key and configuration values for the Firebase Realtime Database.

Below is a list of the packages installed in the backend, with a summary of their functionality.

@google/maps The Node.js Client for Google Maps Services, for querying the Directions API and the Roads API.
@mapbox/polyline A polyline decoder.
asyncawait A co-routine based polyfill for ECMAScript Async functions.
bluebird A JavaScript promise library.
csv-parse A parser for the comma-separated data in the GTFS data files.
firebase-admin The Firebase Admin SDK for Node.js.
moment A lightweight JavaScript date library.
sqlite3 A database engine for building and querying an in-memory SQL database, storing data from the GTFS timetable.