GVR FMOD Plugins User Guide

The Google VR Audio FMOD integration is a plugin package that enables high- quality spatial audio on mobile (Daydream, Android, iOS) and desktop (OS X, Windows, Linux) VR platforms.

The GVR FMOD Plugins are included as standard with FMOD Studio starting with version 1.09.

The package consists of three audio plugins:

GVR Listener is a FMOD::DSP plugin that allows you to binaurally spatialize sound sources created using either the GVR Source or GVR Soundfield plugins. This plugin also handle room effects simulation.

GVR Source is a FMOD::DSP plugin that can be used to spatialize mono sound objects anywhere in space around the listener. This plugin applies effects such as distance attenuation and directivity patterns.

GVR Soundfield is a FMOD::DSP plugin that can be used to playback first order Ambisonic soundfields reacting to listener rotation.

Using the GVR Audio Listener plugin in FMOD Studio

  1. Create a new empty project in FMOD Studio.

  2. Create a new event in the Events window by right clicking (command click on Mac) in the Events pane to the far left.

  3. Once this event has been created click on the FMOD default 3D panner (it is in the master track of the event) and delete it from the event.

  4. Open the Mixer window (Ctrl/Cmd+2) to set up the GVR Audio Listener plugin by selecting Add Effect -> Plug-in Effects -> Google -> GVR Listener.

Once the listener is added, you can follow the same steps above to add as many events (sources) as you wish to your project.

The GVR Audio Listener plugin can be placed on any mixer. For example, to spatialize only a portion of your projects sources, you can create a group of spatialized events by right-clicking in the left-hand pane of the mixer window. The GVR Audio Listener plugin can be then placed on the deck of that groups' mixer.

In most cases, it's best to place the GVR Audio Listener Plugin on the deck of the master bus. We recommend you place it before the fader so that the fader can still be used to control the output of the listener plugin.

GVR Audio Listener plugin features

1) Global Gain: Controls the gain applied to all GVR sources in your project.

2) Binaural Level Meter: Level meter indicates the output levels of your overall binaural mix.

Using the GVR Audio Source plugin in FMOD Studio

  1. Create a new empty project in FMOD Studio.
  2. Create a new event in the Events window by right clicking (command click on Mac) in the Events pane to the far left.
  3. Once this event has been created click on the FMOD default 3D panner (it is in the master track of the event) and delete it from the event.
  4. Click on the master track of your new event and add the GVR Audio Source Plugin to the master tracks deck by selecting Add Effect -> Plug-in Effects -> Google -> GVR Source.

    You can also choose to spatialize only some of the audio tracks for an event, by placing a GVR Source Plugin in the deck for just one of the audio tracks. This will direct the audio from that track into the GVR audio system and away from the events master track.

    In most cases, we recommend that the GVR Audio Source Plugin be placed in the deck of the events master track.

Audio Source plugin features

1) Distance attenuation curve: Indicates the shape of the distance attenuation curve as a source moves from or towards the listener.

2) Curve Type Selector: Choose between a linear distance attenuation curve, a logarithmic attenuation curve, or no distance attenuation effect.

3) Source Gain: Gain applied to the given GVR Audio Source.

4) Source Directivity Curve: This curve shows the directivity of the given source. This indicates the degree to which a source will emit sounds in different directions. A circular curve indicates the source will emit sound uniformly in all directions. A source with a figure-eight pattern would emit sound from the front and back but not from the sides.

5) Minimum and Maximum Distance selector: This slider can be used to set the minimum and maximum source distances. These are the distances at which the source is heard at full volume (minimum distance), and the distance at which the source falls completely silent (maximum distance). The curve shown above (1) indicates the rate of volume falloff with distance, given these two parameters.

6) Bypass Room Effects: If GVR Audio room effects have been applied to your project, you can choose for those room effects not to apply to any given source by enabling this switch.

7) Spread: This parameter approximately defines the source width of the given GVR Audio Source in degrees. Higher values correspond to wider, less point-like sources.

8) Source Occlusion: Increasing this parameter will remove high frequencies from a source, attenuating the source in doing so. This simulates the effect of a source being occluded by an object between it and the listener.

9) Directivity Parameter: Increasing this parameter morphs the directivity curve (4) from a circular pattern, through a cardioid pattern, into a figure-eight pattern.

10) Directivity Sharpness: Increasing this parameter decreases the width of the lobes of the directivity curve (4). This can be used to create narrower emission patterns for your sound sources.

Using the Soundfield Plugin in FMOD Studio

First-order Ambisonic soundfields (FOA) can be played back using the GVR Audio Soundfield plugin. This plugin supports first order Ambisonics encoded in the AmbiX (ACN-SN3D) format. The same format is supported by YouTube. GVR Audio Soundfields react to rotation but not to translational movement.

FMOD upmixes four channel busses on the master track to 5.1 surround format, which is unsuitable for Ambisonic playback. In order to ensure that an Ambisonic sound file is played back correctly, there are two options:

  1. Ensure that the GVR Audio Soundfield plugin is placed on the same audio track as the audio file and not on the master track of an event within FMOD Studio. This ensures that the input to the plugin is 4-channel FOA audio. Note that it is not possible to audition soundfield rotation in FMOD Studio with this method.

  2. If you want to mix multiple Ambisonic soundfields on the same event you can modify the audio files themselves such that FMOD will not up-mix to 5.1. To do this, insert a pair of silent channels into your FOA file. For example, if you have a soundfield with channels X[0, 1, 2, 3], you can map it to a six-channel file Y[0,1,2,3,4,5] in the following manner:

    1. Y[0] = X[0] (ACN channel 0)
    2. Y[1] = X[1] (ACN channel 1)
    3. Y[2] = ZEROS
    4. Y[3] = ZEROS
    5. Y[4] = X[2] (ACN channel 2)
    6. Y[5] = X[3] (ACN channel 3)

    Once this is done, you can place different Ambisonic soundfields, modified in this manner, on separate tracks of an FMOD Event. The GVR Audio Soundfield plugin can then be placed on the 5.1 master track and will know to interpret the six-channel input as FOA data.

Listed below is a Ffmpeg command that can be run on any four-channel FOA sound file in order to make the necessary conversion. Here, ‘input.wav’ represents a four channel FOA input file and ‘output.wav’ represents the six-channel output containing the two zeroed out channels:

ffmpeg -i input.wav -map_channel 0.0.0 -map_channel 0.0.1 -map_channel -1 -map_channel -1 -map_channel 0.0.2 -map_channel 0.0.3 output.wav

Room effects

The GVR Audio Room Effects plugin uses the plugin custom game data structure in the FMOD SDK to set the room properties in the environment. These room properties can be set via the FMOD::DSP::setParameterData method in the game engine integration of your project. Here the data should be passed in as a void pointer.

FMOD_RESULT DSP::setParameterData(
  int index,
  void *data,
  unsigned int length
);

To set the RoomProperties Struct to control room effects, the first index parameter should be passed a value of 1. For instance,

setParameterData( 1 /* Room properties index */, room_properties_ptr, length_bytes);

The RoomProperties struct header is available on github.

Game engine integration

Unity integration

The GVR Audio plugins can be used as a part of any FMOD Unity project. Within Unity the GVR Audio plugins provide an advanced set of room effects that provides fine control over the room sound and reacts to the listener’s position in space.

Once you have chosen your FMOD project from within Unity by navigating to FMOD -> Edit Settings and choosing an FMOD Studio project, you must then click the Add Plugin radio button towards the bottom of the inspector window and type ‘gvraudio’ into the provided text box.

The FMOD Studio Unity Integration package contains scripts needed to access GVR Audio Room Effects. Once you've imported the package into your Unity project, the GVR plugins are available in the folder GoogleVR.

If you choose to use this feature, we recommend you remove the FMOD Reverb return effect from the master. This can be done by doing the following:

  1. Navigate to the Mixer window (Ctrl/Cmd+2).
  2. Select the reverb return from the Routing menu on the far left.
  3. Remove by pressing the back-space key, or by right/command-clicking and choosing 'delete'.

Room effects in Unity

Once you have completed the steps above and have sources using the GVR Audio spatialization within your project you can use the provided C# scripts: “FmodGvrAudio.cs, FmodGvrAudioRoom.cs, FmodGvrAudioRoomEditor.cs” to control room effects in your Unity project.

  1. Add the GoogleVR folder into ($UnityProject)/Assets/ folder.
  2. Attach the FmodGvrAudioRoom component to a GameObject in your scene. Then, adjust its properties as desired (both the audio listener, and the audio sources should be inside the selected room boundaries for the corresponding room effects to take effect).

Any number of GVR rooms can be placed in a scene. As the game object with the FMOD listener script attached moves about the scene the room effects will react smoothly in real time, updating based upon the room, the listener and any GVR sound sources that are currently in.

The character of any room can be updated and altered to match the exact sound you desire using the following controls:

  1. Surface Materials Drop Down Menus: Each of the six surfaces of the GVR Audio Room can be assigned a surface material type from its drop down menu. Each material has different degrees of absorption or reflectivity at different frequencies. Some materials such as ’Heavy Curtain’ absorb most high frequencies giving rooms a dryer warmer sound, while materials like ‘Polished Concrete’ reflect a greater proportion of sound energy at all frequencies resulting in much brighter and more echoic room characteristics.

  2. Reflectivity Parameter: This parameter gives the user greater control over the strength of the early reflections in a GVR Audio Room. This is particularly useful for very small spaces (such as closets etc) where a user can uses this parameter to achieve the claustrophobic sounds of an enclosed space.

  3. Reverb Gain: The following three parameters affect the late reverberation in the GVR Audio Room. The reverb gain allows the user to control how loud the room effects are compared to the direct sounds coming from GVR Audio sources in their scene. This can be thought of as the wet/dry mix.

  4. Reverb Brightness: The GVR Audio Room effects provide different reverb decay rates at different frequencies, just like in real rooms. The reverb brightness provides users with the ability to carefully balance the amount of low or high frequencies in their reverb. One particular use of this parameter is to control how full a room sounds. By reducing the reverb brightness a user can give the impression that a room is less empty and simulate the sounds of a room filled with furniture or people.

  5. Reverb Time: This parameter allows users to increase or decrease the length of their reverb. The value is a multiplier on the reverb time calculated from the surface materials and room dimensions. Users can take advantage of this parameter to reduce or increase the echoic nature of their rooms as they see fit allowing a high degree of user control.

  6. Room Size: These x,y,z parameters allow users to set the dimensions of their given GVR Audio Room. It is highly recommended that users use these size parameters over the ‘scale’ of the game object to which the GVR Audio Room is attached to set their room dimensions. Not only do the room dimensions set here affect the sound of the given room, but also set the boundaries, which if crossed, will trigger the room effects to smoothly transition between on and off or from one GVR Audio Room to another.

Deploying the plugins on iOS using Unity

To deploy on iOS, in addition to the above steps, you must modify the file named ($UnityProject)/Assets/Plugins/Fmod/fmodplugins.cpp to declare the following at the top of that file:

extern "C" FMOD_DSP_DESCRIPTION* FMOD_Google_GVRListener_GetDSPDescription();
extern "C" FMOD_DSP_DESCRIPTION* FMOD_Google_GVRSoundfield_GetDSPDescription();
extern "C" FMOD_DSP_DESCRIPTION* FMOD_Google_GVRSource_GetDSPDescription();

and to contain the following within the FmodUnityNativePluginInit() method, which itself is defined in that file:

uint32_t result = 0;
result = FMOD5_System_RegisterDSP(system, FMOD_Google_GVRSource_GetDSPDescription(), nullptr);
if (result != 0)
{
  return result;
}
result = FMOD5_System_RegisterDSP(system, FMOD_Google_GVRListener_GetDSPDescription(), nullptr);
if (result != 0)
{
  return result;
}
result = FMOD5_System_RegisterDSP(system, FMOD_Google_GVRSoundfield_GetDSPDescription(), nullptr);
if (result != 0)
{
  return result;
}

Unreal Integration

To use the GVR Audio FMOD plugins in Unreal Engine do the following:

  1. Move the GVR Audio binaries for the relevant platforms into one of the following directories:

    • Your Unreal project's Plugins/FMODStudio/Binaries/<platform> directory.
    • (Mac) /Users/Shared/UnrealEngine/<version>/Engine/Plugins/FMODStudio/Binaries/<platform>.
    • (Windows) <UnrealEngineRootDir>/Engine/Plugins/FMODStudio/Binaries/<platform>.

  2. Within your FMOD Studio project, make sure that you are building your banks into your Unreal Engine project’s Content/FMOD directory and that you have added 'FMOD' to ‘Additional Non-Asset Directories to Package’ in Unreal Engine under Edit > Project Settings > Packaging.

  3. Within UE4 go to Edit > Project Settings... Scroll to FMOD Studio settings in the lower left hand column and select “FMOD Studio”.

  4. Navigate to the advanced tab and add a plugin file named 'gvraudio'.

  5. You should now be able to load and play FMOD events utilizing the GVR audio plugins.

  6. Create a file named plugins.txt within your game folder under Plugins/FMODStudio/Binaries/<platform> for each platform you wish to deploy to. Each file should contain the name of the plugins you are using. To use the GVR Audio plugins the file only needs to include the word 'gvraudio'.

Deploying the plugins on Android using Unreal

To deploy on Android, in addition to the above steps, you must include a file named gvraudio_APL.xml in Plugins/FMODStudios/Binaries/Android/ with the following content:

<?xml version="1.0" encoding="utf-8"?>
<!--Plugin additions-->
<root xmlns:android="http://schemas.android.com/apk/res/android">
  <!-- init section is always evaluated once per architecture --> 
  <init>
    <log text="gvraudio APL init"/> </init>

<!-- optional files or directories to copy to Intermediate/Android/APK -->
<resourceCopies>
  <log text="gvraudio APL copying files for $S(Architecture)/"/> 
  <copyFile src="$S(PluginDir)/$S(Architecture)/libgvraudio.so"
            dst="$S(BuildDir)/libs/$S(Architecture)/libgvraudio.so" /> 
</resourceCopies>

<!-- optional libraries to load in GameActivity.java before libUE4.so -->
<soLoadLibrary>
  <log text="gvraudio APL adding loadLibrary references"/>
  <loadLibrary name="gvraudio" failmsg="gvraudio not loaded and required!" />
</soLoadLibrary> </root>