AudioData

public class AudioData

Defines a ring buffer and some utility functions to prepare the input audio samples.

It maintains a Ring Buffer to hold input audio data. Clients could feed input audio data via `load` methods and access the aggregated audio samples via `getTensorBuffer` method.

Note that this class can only handle input audio in Float (in ENCODING_PCM_16BIT) or Short (in ENCODING_PCM_FLOAT). Internally it converts and stores all the audio samples in PCM Float encoding.

Typical usage in Kotlin

  val audioData = AudioData.create(format, modelInputLength)
  audioData.load(newData)

Another sample usage with android.media.AudioRecord

  val audioData = AudioData.create(format, modelInputLength)
  Timer().scheduleAtFixedRate(delay, period) {
    audioData.load(audioRecord)
  }

Summary

Nested types

public abstract class AudioData.AudioDataFormat

Wraps a few constants describing the format of the incoming audio samples, namely number of channels and the sample rate.

public abstract class AudioData.AudioDataFormat.Builder

Builder for AudioDataFormat

Public fields

final AudioData.AudioDataFormat

Public methods

static AudioData
create(AudioData.AudioDataFormat format, int sampleCounts)

Creates a android.media.AudioRecord instance with a ring buffer whose size is sampleCounts * format.getNumOfChannels().

static AudioData
create(AudioFormat format, int sampleCounts)

Creates a AudioData instance with a ring buffer whose size is sampleCounts * format.getChannelCount().

static AudioData
create(
    AudioData.AudioDataFormat format,
    ByteBuffer buffer,
    int sampleCounts
)

Creates an AudioData instance wrapping a pre-allocated direct ByteBuffer.

float[]

Returns a float array holding all the available audio samples in ENCODING_PCM_FLOAT i.e. values are in the range of [-1, 1].

int
ByteBuffer

Returns a ByteBuffer containing the ordered audio float samples.

AudioData.AudioDataFormat

Returns the AudioDataFormat associated with the tensor.

int
load(AudioRecord record)

Loads latest data from the android.media.AudioRecord in a non-blocking way.

void
load(float[] src)

Stores the input audio samples src in the ring buffer.

void
load(short[] src)

Converts the input audio samples src to ENCODING_PCM_FLOAT, then stores it in the ring buffer.

void
load(float[] src, int offsetInFloat, int sizeInFloat)

Stores the input audio samples src in the ring buffer.

void
load(short[] src, int offsetInShort, int sizeInShort)

Converts the input audio samples src to ENCODING_PCM_FLOAT, then stores it in the ring buffer.

Public fields

format

public final AudioData.AudioDataFormat format

Public methods

create

public static AudioData create(AudioData.AudioDataFormat format, int sampleCounts)

Creates a android.media.AudioRecord instance with a ring buffer whose size is sampleCounts * format.getNumOfChannels().

Parameters
AudioData.AudioDataFormat format

the expected AudioDataFormat of audio data loaded into this class.

int sampleCounts

the number of samples.

create

public static AudioData create(AudioFormat format, int sampleCounts)

Creates a AudioData instance with a ring buffer whose size is sampleCounts * format.getChannelCount().

Parameters
AudioFormat format

the android.media.AudioFormat required by the TFLite model. It defines the number of channels and sample rate.

int sampleCounts

the number of samples to be fed into the model

create

public static AudioData create(
    AudioData.AudioDataFormat format,
    ByteBuffer buffer,
    int sampleCounts
)

Creates an AudioData instance wrapping a pre-allocated direct ByteBuffer.

getBuffer

public float[] getBuffer()

Returns a float array holding all the available audio samples in ENCODING_PCM_FLOAT i.e. values are in the range of [-1, 1].

getBufferLength

public int getBufferLength()

getByteBuffer

public ByteBuffer getByteBuffer()

Returns a ByteBuffer containing the ordered audio float samples.

getFormat

public AudioData.AudioDataFormat getFormat()

Returns the AudioDataFormat associated with the tensor.

load

public int load(AudioRecord record)

Loads latest data from the android.media.AudioRecord in a non-blocking way. Only supporting ENCODING_PCM_16BIT and ENCODING_PCM_FLOAT.

Parameters
AudioRecord record

an instance of android.media.AudioRecord

Returns
int

number of captured audio values whose size is channelCount * sampleCount. If there was no new data in the AudioRecord or an error occurred, this method will return 0.

Throws
java.lang.IllegalArgumentException

for unsupported audio encoding format

java.lang.IllegalStateException

if reading from AudioRecord failed

load

public void load(float[] src)

Stores the input audio samples src in the ring buffer.

Parameters
float[] src

input audio samples in ENCODING_PCM_FLOAT. For multi-channel input, the array is interleaved.

load

public void load(short[] src)

Converts the input audio samples src to ENCODING_PCM_FLOAT, then stores it in the ring buffer.

Parameters
short[] src

input audio samples in ENCODING_PCM_16BIT. For multi-channel input, the array is interleaved.

load

public void load(float[] src, int offsetInFloat, int sizeInFloat)

Stores the input audio samples src in the ring buffer.

Parameters
float[] src

input audio samples in ENCODING_PCM_FLOAT. For multi-channel input, the array is interleaved.

int offsetInFloat

starting position in the src array

int sizeInFloat

the number of float values to be copied

Throws
java.lang.IllegalArgumentException

for incompatible audio format or incorrect input size

load

public void load(short[] src, int offsetInShort, int sizeInShort)

Converts the input audio samples src to ENCODING_PCM_FLOAT, then stores it in the ring buffer.

Parameters
short[] src

input audio samples in ENCODING_PCM_16BIT. For multi-channel input, the array is interleaved.

int offsetInShort

starting position in the src array

int sizeInShort

the number of short values to be copied

Throws
java.lang.IllegalArgumentException

if the source array can't be copied