Spatial Audio for Android

The GvrAudioEngine class provides an interface to JNI routines for the GVR Audio engine library.

public class GvrAudioEngine {}

All of the code detailed on this page is contained within this class.

GvrAudioEngine allows the user to spatialize sound sources in 3D space, including distance and height cues. It is capable of playing back spatial sound in three separate ways:

The first method, known as Sound Object rendering, allows the user to create a virtual sound source in 3D space. These sources, while spatialized, are fed with mono audio data.

The second method allows the user to play back Ambisonic soundfield recordings. Ambisonic recordings are multi-channel audio files that are spatialized all around the listener in 360 degrees. These can be thought of as recorded or prebaked soundfields. They can be of great use for creating spatialized background effects. Examples include rain, crowd noise, or even the sound of the ocean off to one side.

The third method, referred to here as Stereo Sounds, allows the user to directly play back non-spatialized mono or stereo audio files. This is useful for music and other audio.

Construction

The RenderingMode class defines constants to adjust rendering mode. Higher quality means more CPU load.

public abstract static class RenderingMode {

  /* Stereo panning of all sounds. This disables HRTF-based rendering. */
  public static final int STEREO_PANNING = 0;

  /* Renders all sounds over eight virtual loudspeakers arranged around
  the listener’s head. HRTF-based rendering is enabled. */
  public static final int BINAURAL_LOW_QUALITY = 1;

  /* Renders all sounds over 16 virtual loudspeakers arranged around
  the listener’s head. HRTF-based rendering is enabled. */
  public static final int BINAURAL_HIGH_QUALITY = 2;

}

Load the native library.

static {
  System.loadLibrary("vraudio_engine");
}

Construct the GVR Audio system with an Android context and RenderingMode.

public GvrAudioEngine(Context context, int renderingMode) {
  vrAudioSystemRef = nativeInitialize(getClass().getClassLoader(),
                                      context.getApplicationContext(),
                                      renderingMode);
}

private native long nativeInitialize(
    ClassLoader appClassLoader, Context applicationContext,
    int renderingMode);

Invalid Source ID.

public static final int INVALID_ID = -1;

Sound engine control

Pause the GVR Audio system.

public void pause()

Resume the GVR Audio system.

public void resume()

The update method must be called from the main thread at a regular rate. It is used to execute background operations outside of the audio thread.

public void update()

Sound files and preloading

Both mono sound files for use with Sound Objects and multi-channel Ambisonic sound files can be preloaded into memory before playback with the PreloadSoundfile() method or alternatively streamed during playback. Preloading can be useful to reduce CPU usage especially if the same audio clip is likely to be played back many times. In this case, playback latency is also reduced. To clean up unused sound files, they can be unloaded with UnloadSoundfile().

Spatialization of Sound Objects

The GVR Audio System allows the user to create virtual Sound Objects that can be placed anywhere in space around the listener. Sound Objects take mono soundfiles as input.

Create a Sound Object

Return a new Sound Object handle. The handle automatically destroys itself at the moment the sound playback has stopped.

public int createSoundObject(String filename)

Sound position and rotation

Reposition an existing Sound Object.

public void setSoundObjectPosition(int soundObjectId, float x,
                                   float y, float z)

Volume rolloff as a sound object moves away from the listener can be set to either a logarithmic rolloff, a linear rolloff or no distance based rolloff at all.

public void setSoundObjectDistanceRolloffModels(
  int soundObjectId, int rolloffModel, float minDistance, float maxDistance)

Rolloff models: full list

public abstract static class DistanceRolloffModel {

  /** Logarithmic distance rolloff model. */
  public static final int LOGARITHMIC = 0;

  /** Linear distance rolloff model. */
  public static final int LINEAR = 1;

  /** No distance rolloff will be applied. */
  public static final int NONE = 2;

}

Render an Ambisonic soundfield

The GVR Audio System provides the user with the ability to play back Ambisonic soundfields. Ambisonic soundfields are captured or pre-rendered 360-degree recordings. It is best to think of them as equivalent to 360-degree video. While they envelop and surround the listener, they only react to the listener's rotational movement. That is, one cannot walk towards features in the soundfield. Soundfields are ideal for accompanying 360-degree video playback, for introducing background and environmental effects such as rain or crowd noise, or even for pre-baking 3D audio to reduce rendering costs. The GVR Audio System supports full 3D First-Order Ambisonic recordings using ACN channel ordering and SN3D normalization. For more information, see our Spatial Audio specification.

Note that the sample must have four audio channels. The handle automatically destroys itself at the moment the sound playback has stopped.

Return a new Ambisonic soundfield handle.

public int createSoundfield(String filename)

Set the rotation of an existing Ambisonic soundfield.

public void setSoundfieldRotation(int soundfieldId, float x,
                                  float y, float z, float w)

Create a Stereo Sound

Return a new non-spatialized Stereo Sound. Note that the sample is limited to a maximum of two audio channels. Both mono and stereo audio files are supported. The handle automatically destroys itself at the moment the sound playback has stopped.

public int createStereoSound(String filename)

Sound playback

Start the playback of a sound.

public void playSound(int sourceID, boolean loopingEnabled)

Check if a sound is playing.

public boolean isSoundPlaying(int sourceID)

Pause the playback of a sound.

public void pauseSound(int sourceID)

Resume the playback of a sound.

public void resumeSound(int sourceID)

Stop the playback of a sound and destroy the corresponding Sound Object or soundfield.

public void stopSound(int sourceID)

Checks that a sourceID corresponds to a valid source which exists and is in a playable state.

public boolean IsSourceIdValid(int sourceID)

Head position and rotation

To ensure that the audio in your application reacts to listener head movement, it is important to update the listener's head position and orientation with the audio engine.

Set the head position.

public void setHeadPosition(float x, float y, float z)

Set the head rotation.

public void setHeadRotation(float x, float y, float z, float w)

Change volume

Change the volume of an existing sound.

public void setSoundVolume(int sourceID, float volume)

Reverberation

Turn on/off the room reverberation effects.

public void enableRoom(boolean enable)

The following method can be used to subtly adjust the reverb in a room by changing the gain/attenuation on the reverb, setting a multiplier on the reverberation time to control the reverb's length, or adjusting the balance between the low and high frequency components of the reverb.

void SetRoomReverbAdjustments(float gain, float time_adjust,
                              float brightness_adjust)

Surface materials

Define constants representing the surface materials for use with room effects. Each material has different absorption properties across frequency bands.

public abstract static class MaterialName {

  /* Acoustically transparent material, reflects no sound. */
  public static final int TRANSPARENT = 0;

  /* Acoustic ceiling tiles, absorbs most frequencies. */
  public static final int ACOUSTIC_CEILING_TILES = 1;

  /* Bare brick, relatively reflective. */
  public static final int BRICK_BARE = 2;

  ............................

}

Room properties

Set the room properties describing the dimensions and surface materials of a given room.

public void setRoomProperties(float sizeY, float sizeZ, int wallMaterial,
                              int ceilingMaterial, int floorMaterial)

Surface materials: full list

public abstract static class MaterialName {

  /* Acoustically transparent material, reflects no sound. */
  public static final int TRANSPARENT = 0;

  /* Acoustic ceiling tiles, absorbs most frequencies. */
  public static final int ACOUSTIC_CEILING_TILES = 1;

  /* Bare brick, relatively reflective. */
  public static final int BRICK_BARE = 2;

  /* Painted brick. */
  public static final int BRICK_PAINTED = 3;

  /* Coarse surface concrete block. */
  public static final int CONCRETE_BLOCK_COARSE = 4;

  /* Painted concrete block. */
  public static final int CONCRETE_BLOCK_PAINTED = 5;

  /* Heavy curtains. */
  public static final int CURTAIN_HEAVY = 6;

  /* Fiberglass insulation. */
  public static final int FIBER_GLASS_INSULATION = 7;

  /* Thin glass. */
  public static final int GLASS_THIN = 8;

  /* Thick glass. */
  public static final int GLASS_THICK = 9;

  /* Grass. */
  public static final int GRASS = 10;

  /* Linoleum on concrete. */
  public static final int LINOLEUM_ON_CONCRETE = 11;

  /* Marble. */
  public static final int MARBLE = 12;

  /** Galvanized sheet metal. */
  public static final int METAL = 13;

  /* Wooden parquet on concrete. */
  public static final int PARQUET_ON_CONCRETE = 14;

  /* Rough plaster surface. */
  public static final int PLASTER_ROUGH = 15;

  /* Smooth plaster surface. */
  public static final int PLASTER_SMOOTH = 16;

  /* Plywood panel. */
  public static final int PLYWOOD_PANEL = 17;

  /* Polished concrete OR tile surface. */
  public static final int POLISHED_CONCRETE_OR_TILE = 18;

  /* Sheetrock. */
  public static final int SHEET_ROCK = 19;

  /* Surface of water or ice. */
  public static final int WATER_OR_ICE_SURFACE = 20;

  /* Wooden ceiling. */
  public static final int WOOD_CEILING = 21;

  /* Wood paneling. */
  public static final int WOOD_PANEL = 22;
}