Spatial Audio

The GVRAudioEngine allows the user to spatialize sound sources in 3D space, including distance and height cues. The GVRAudioEngine is capable of playing back spatial sound in two separate ways:

The first method, known as Sound Object rendering, allows the user to create a virtual sound source in 3D space. These sources, while spatialized, are fed with mono audio data.

The second method allows the user to play back Ambisonic soundfield recordings. Ambisonic recordings are multi-channel audio files which are spatialized all around the listener in 360 degrees. These can be thought of as recorded or prebaked soundfields. They can be of great use for background effects which sound perfectly spatial. Examples include rain noise, crowd noise or even the sound of the ocean off to one side.

Construction

- (id)initWithRenderingMode: (GVRAudioEngine)rendering_mode;

Alternatively, using initWithRenderingMode without parameters will default to binaural high quality mode.

RenderingMode is an enum which specifies a global rendering configuration setting:

  • kRenderingModeStereoPanning: Stereo panning of all Sound Objects. This disables HRTF-based rendering.

  • kRenderingModeBinauralLowQuality: This renders Sound Objects over a virtual array of eight loudspeakers arranged in a cube about the listener’s head. HRTF-based rendering is enabled.

  • kRenderingModeBinauralHighQuality: This renders Sound Objects over a virtual array of 16 loudspeakers arranged in an approximate equidistribution about the listener’s HRTF-based rendering is enabled.

If ARC is not enabled, a call to the dealloc method must be made. See the Example Usage snippet below.

Audio playback can be started and stopped by calling the following two methods:

- (bool)start;
- (void)stop;

Sound files

Both mono sound files for use with Sound Objects and multi-channel Ambisonic sound files can be preloaded with a call to the following method:

- (bool)preloadSoundFile: (GVRAudioEngine)filename;

Sound objects

The GVRAudioEngine allows the user to create virtual Sound Objects which can be placed anywhere in space around the listener. These Sound Objects take as input mono audio data which is then spatialized.

Once a suitable audio file is preloaded, it can be played back in 3D space with a call to the following function:

- (int)createSoundObject: (GVRAudioEngine)filename;

Here the filename serves as a handle on the preloaded audio file.

This method returns an int handle which can be used to refer to the Sound Object as it is manipulated.

Playback of a Sound Object can be initiated with a call to:

- (void)startSound:(int)soundId loopingEnabled:(bool)loopingEnabled;

The loopingEnabled boolean allows the user to specify whether the sound source should repeat continuously or should be played as a "single shot."

A Sound Object’s position in space or loudness can be altered by calling the following methods:

- (void)setSoundObjectPosition:(int)soundObjectId
                             x:(float)x
                             y:(float)y
                             z:(float)z;

The three variables x, y, z denote the position in Cartesian world space coordinates at which the Sound Object shall appear.

The behavior of Sound Objects with respect to their distance from the listener can be controlled via calls to the following method:

- (void)setSoundObjectDistanceRolloff:(int)soundObjectId
                         rolloffModel:(distanceRolloffModel)rolloffModel
                          minDistance:(float)minDistance
                          maxDistance:(float)maxDistance;

This enables a user to choose between logarithmic and linear distance rolloff methods, or to completely disable distance rolloff effects.

The volume variable allows the user to control the loudness of individual sources. This can be useful when some of your mono audio files are intrinsically louder than others. A value of 1.0f ensures that the mono audio amplitude is not modified. This is controlled via the following method:

- (void)setSoundVolume:(int)soundId volume:(float)volume;

Caution is encouraged when using very loud (e.g. 0dB FS normalized) mono audio data, audio data that has been heavily dynamic range-compressed, or when using multiple sources. In order to avoid clipping, individual sound object volumes can be reduced by calling setSoundVolume() method.

The user can ensure that the Sound Object is currently playing before calling the above methods with a call to:

- (bool)isSoundPlaying: (GVRAudioEngine)soundId;

Once one is finished with a sound object and wishes to remove it, simply place a call to:

- (void)stopSound: (GVRAudioEngine)soundId;

On making a call to this function the Sound Object is destroyed and the corresponding integer handle no longer refers to a valid Sound Object.

Ambisonic soundfields

The GVRAudioEngine is also designed to play back Ambisonic soundfields. These are captured or pre-rendered 360 degree recordings. It is best to think of them as equivalent to 360 degree video. While they envelop and surround the listener, they only react to rotational movement of the listener. That is, one cannot walk towards features in the soundfield. Soundfields are ideal for accompanying 360 degree video playback, for introducing background and environmental effects such as rain or crowd noise, or even for prebaking 3D audio to reduce rendering costs. The GVRAudioEngine supports full 3D First-Order Ambisonic recordings using ACN channel ordering and SN3D normalization. For more information, see our Spatial Audio specification.

A preloaded multi channel Ambisonic sound file can be used to create a soundfield with a call to:

- (int)createSoundfield: (GVRAudioEngine)filename;

Once again an integer handle is returned allowing the user to begin playback of the soundfield, alter the soundfield’s volume, or stop soundfield playback and as such destroy the object.

- (void)startSound:(int)soudnObjectId loopingEnabled:(bool)loopingEnabled;
- (void)setSoundVolume:(int)soundId volume:(float)volume;
- (void)stopSound: (GVRAudioEngine)soundId;

Paused Sounds and Stopped Sounds

When using sound sources of any of the above types, the user can ensure that the given source is currently playing before calling.

- (bool)isSoundPlaying:(int)sourceId;

This method will return false if the source has been either paused or stopped, and true if the source is currently playing.

Once one is finished with a Sound Object and wish to remove it, a call can be placed to:

- (void)stopSound:(int)sourceId;

Once a source has been stopped it is destroyed and the corresponding SourceId will be invalid. Sources which have been played with the |looping_enabled| parameter disabled will also be destroyed once playback of the full audio clip has completed.

To check whether a given SorceId corresponds to a valid source which exists and is in a playable state, a call can be made to:

- (bool)isSourceIdValid:(int)sourceId;

By using this pair of methods a user can differentiate between sources which have been paused and those which have ceased.

Listener position and rotation

In order to ensure that the audio in your application reacts to user head movement it is important to update head orientation in the graphics callback using the head orientation matrix.

The following two methods control the listener’s head orientation in terms of audio:

- (void)setHeadPosition:(float)x y:(float)y z:(float)z;

... where x, y and z are cartesian world space coordinates, and

- (void)setHeadRotation:(float)x y:(float)y z:(float)z w:(float)w;

Here x, y, z and w are the components of a quaternion.

Room effects

The GVRAudioEngine provides a reverb engine enabling the user to create arbitrary room effects by specifying the size of a room and a material for each surface of the room from the SurfaceMaterials enum. Each of these surface materials has unique absorption properties which differ with frequency. The room created will be centered around the listener. Note that in the GVRAudioEngine the unit of distance is meters.

The following methods are used to control room effects:

- (void)enableRoom: (GVRAudioEngine)enable

... which enables or disables room effects with smooth transitions, and

- (void)setRoomProperties:(float)size_x
                   size_y:(float)size_y
                   size_z:(float)size_z
            wall_material:(MaterialName)wall_material
         ceiling_material:(MaterialName)ceiling_material
           floor_material:(MaterialName)floor_material;

... which allows the user to describe the room based on its dimensions (size_x, size_y, size_z), and its surface properties. For example, one can expect very large rooms to be more reverberant than smaller rooms and for a room with brick surfaces to be more reverberant than one with heavy curtains on every surface.

Note that sources located outside of a room will sound quite different from those inside due to an attenuation of reverb and direct sound while sources far outside of a room will not be audible.

Example usage:

Initialize a GVRAudioEngine in binaural high quality rendering mode:

GVRAudioEngine *gvrAudio;
gvrAudio = [[GVRAudioEngine alloc]
                  initWithRenderingMode:kRenderingModeBinauralHighQuality];

Load an audio file (compressed or uncompressed) from the main bundle:

NSString filename = "mono_audio_file.mp3";
bool filePreloaded = [gvrAudio preloadSoundFile:filename];

Start audio playback:

bool playbackStarted = [gvrAudio start];

Create a Sound Object with the preloaded audio file:

int soundId = -1;
if(filePreloaded) {
  soundId = [gvrAudio createSoundObject:filename]; }

Begin Playback of the Sound Object:

if (soundId != -1) {
  [gvrAudio startSound:soundId loopingEnabled:true]; }

Change the location and volume of the Sound Object:

if(soundId != -1) {
  [gvrAudio setSoundObjectPosition:soundId x:0.5f y:2.0f z:1.2f];
  [gvrAudio setSoundVolume:0.75f]; }

Change the listener position:

[gvrAudio setHeadPosition:0.5f y:0.5f z:0.5f];

Stop playback of the preloaded audio file:

if([gvrAudio isSoundPlaying:soundId]) {
  [gvrAudio stopSound:soundId]; }

Stop audio playback:

[gvrAudio stop];

If ARC is not enabled:

[gvrAudio dealloc];