Using Sound

A critical part of immersing viewers in your story is the sound they experience around them. Google Spotlight Stories supports a rich set of functions to include music and sound effects and adjust them, or even move them around, as the story progresses.

This page discusses the basics of using sound in your Google Spotlight Story.

Adding a Sound

Adding ambient sounds to a story is simple. The SDK offers a Play Sound command in which you enter the sound file you want to play.

Using Sound Commands

There are several sound-related commands you can include in your story to control the playback of audio clips. You can add the commands either to a state (using an action) or to a transition.

Sound Action Menu

  • Play Sound starts an audio clip in your story.
  • Set Pan adjusts the balance of an audio clip.
  • Set Volume changes the volume of an audio clip.
  • Stop Sound stops a playing audio clip.

These commands are described in more detail in Sound Commands.

When you add a sound using the Play Sound command, an audio clip is created and is listed (along with its properties) in the Audio Clips section of the Outliner.

Sound Files

Google Spotlight Stories can use mono, stereo, Ambisonic and quad sound files, depending on the emitter types you are using. Sound files are converted to the .ogg format during the build process.

We recommend starting with 16-bit, 48kHz .wav files. Files that use other bit rates will be converted, which may degrade sound quality.

We also recommend that you store all of your audio files in subdirectories of your story folder (for example in /sound/MusicAssets/ and /sound/positionalFX/) to make them easier to find when browsing files.

Using Audio Emitters

By default, Stereo and Mono clips will play in normal stereo channels, without regard to the viewer's camera orientation. There is no apparent direction to them. However, if you want to have your sound seem to come from a particular position, you use an emitter.

Emitters are locators you add to your Maya scene. You can name them, position them, and can even (indirectly) animate them.

When you add a Play Sound command to your story, you can choose from a pull-down list of emitters in your Maya scene. Then, when the story is baked (rendered) and played back, the sound will seem to come from that emitter's location.

Types of Emitters

Currently the types of emitters you can use include: 2D, 3D, Surround, Quad, and Stereo Headlock. See Audio Emitters for details.

Adding Emitters in Maya

To add an emitter to your Maya scene, click on the Create Audio Emitter button in the Spotlight Stories shelf.

Emitter Icon

You can then position the emitter in 3D space based on where you want a sound to come from. They will not appear in your final scene renders.

Emitter Object

We recommend using individual emitters for the following:

  • One emitter for each actor that has an associated sound. (Sometimes you may want different emitters for different parts of a large or complex actor.)
  • An emitter for Ambisonic surround sound.
  • An emitter for Quad sound if you are using it.
  • Additional emitters for environment-specific sound locations.

Namespaces in Maya

If you are going to use an emitter across multiple scenes, be aware that you should add the emitter directly into a Maya scene, not reference it in with, for example, a character (actor). A referenced emitter used across multiple scenes will import into the SDK as different emitters with different names for each scene. That means you will not be able to play a sound continuously from that emitter across those scenes.

Setting Emitter Properties in the SDK

When you import your scene, all of the emitters added to your Maya scene are listed in the Outliner window of the Story Editor. Under Sources, click to expand your Maya scene, and then click on Audio Emitters.

Click on an Audio Emitter name to bring up its Properties. There you can choose what type of emitter it will be and choose its attributes. For more information, see Audio Emitters on the Outliner page.

Sound Interactivity

Looping

To loop an audio clip, click on the Looping check box in the clip's properties.

It is good practice to do two things to an audio loop:

  1. Fade out the audio loop using the Set Volume command. This avoids having the sound finish abruptly, which might be jarring to the viewer.
  2. Use the Stop Sound command to end the sound after fading out. This not only frees up the resources used to play the sound, but prevents the sound from possibly continuing to play, even at zero volume, beyond the end of the story.

Quantization Basics

You can set up an audio clip to start playing in time with the beat of another master audio clip. For example, you may have dialog or sound effects that you want to start on the beat of the background music. To do this, you enable Quantization by clicking on the checkbox in the Audio Clip properties.

You can see the beats of the audio clip in the waveforms shown in the Properties window. Each waveform represents one channel of the clip. Once you enable Quantization, you will also see vertical lines in the waveforms. You can adjust the position of the first vertical line (the Offset) and the interval between them (the Pulse).

Audio Waveforms

The audio clip that you want to align with, for example the background music, is called the master. The master clip must have Quantization enabled as well, with its pulse interval set to match its beat. Quantization works by aligning the first beat of a clip with a pulse from its master clip.

In the properties of the Play Sound command that you use to play the audio clip, you can specify the Quantization Master clip. Note that if the master is not playing when you start the audio clip, it will start normally and not use the Quantization settings.

You can also click on the waveform (away from the pulse lines) and drag it to scroll left and right. Use the mouse wheel to zoom in on the clip. If the pulse lines get too close together, they will become unselectable (grayed out). Zoom out to make them selectable again.

Angular and Distance Rolloff

You can change the levels of audio playback based on the angle and distance of the camera to an emitter. These are called Angular Rolloff and Distance Rolloff. They are described in more detail in the Audio Emitters section of the Outliner page.

Using Occluders

Audio Occluders are objects (plane segments) you add to your Maya scene. When they are positioned between the viewer and an audio emitter, they will reduce the perceived level of the audio clip coming from that emitter. (Specifically, the amount of occlusion reduces the cutoff frequency of a lowpass filter.)

You can change the size, position, and orientation of occluder planes in Maya, and even animate them.

To add an occluder to your Maya scene, click on the Create Audio Occluder button in the Spotlight Stories shelf.

Occluder Icon

You can then position the occluder in 3D space. It will not appear in your final scene renders.

Occluder Object

For information on specific attributes, see the Audio Occluders section of the Outliner page.

Animating Emitters and Occluders in Maya

You can not set keyframes on an emitter or occluder, or its associated transform, directly. To animate an emitter or occluder, it is usually easiest to constrain it to an actor in the scene, for example a joint, point on the mesh, group node, or another locator. Then the emitter or occluder will follow the object through the scene. During the bake process, emitters and occluders are recognized and exported as animated audio emitters and occluders.