AudioEmbedder

public final class AudioEmbedder

Performs audio embedding extraction on audio clips or audio stream.

This API expects a TFLite model with mandatory TFLite Model Metadata that contains the mandatory AudioProperties of the solo input audio tensor and the optional (but recommended) label items as AssociatedFiles with type TENSOR_AXIS_LABELS per output classification tensor.

Input tensor: (kTfLiteFloat32)

• input audio buffer of size `[batch * samples]`.
• batch inference is not supported (`batch` is required to be 1).
• for multi-channel models, the channels need be interleaved.

At least one output tensor with: (kTfLiteFloat32)

• `N` components corresponding to the `N` dimensions of the returned feature vector for this output layer.
• Either 2 or 4 dimensions, i.e. `[1 x N]` or `[1 x 1 x 1 x N]`.

Nested Classes

class

AudioEmbedder.AudioEmbedderOptions

Options for setting up and AudioEmbedder.

Public Methods

static AudioEmbedder	createFromBuffer(Context context, ByteBuffer modelBuffer) Creates an AudioEmbedder instance from a model buffer and default AudioEmbedder.AudioEmbedderOptions.
static AudioEmbedder	createFromFile(Context context, String modelPath) Creates an AudioEmbedder instance from a model file and default AudioEmbedder.AudioEmbedderOptions.
static AudioEmbedder	createFromFile(Context context, File modelFile) Creates an AudioEmbedder instance from a model file and default AudioEmbedder.AudioEmbedderOptions.
static AudioEmbedder	createFromOptions(Context context, AudioEmbedder.AudioEmbedderOptions options) Creates an AudioEmbedder instance from an AudioEmbedder.AudioEmbedderOptions instance.
AudioEmbedderResult	embed(AudioData audioClip)
void	embedAsync(AudioData audioBlock, long timestampMs)

Inherited Methods

From class com.google.mediapipe.tasks.audio.core.BaseAudioTaskApi

void	close() Closes and cleans up the MediaPipe audio task.
AudioRecord	createAudioRecord() Creates an AudioRecord instance to record audio stream that has mono channel at sample rate at sample rate 16kHz, the sample rate required for models like Yamnet.
AudioRecord	createAudioRecord(int numChannels, int sampleRate, int requiredInputBufferSize) Creates an AudioRecord instance to record audio stream.

From class java.lang.Object

boolean	equals(Object arg0)
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

From interface java.lang.AutoCloseable

abstract void

close()

Public Methods

public static AudioEmbedder createFromBuffer (Context context, ByteBuffer modelBuffer)

Creates an AudioEmbedder instance from a model buffer and default AudioEmbedder.AudioEmbedderOptions.

Parameters

context	an Android ERROR(/Context).
modelBuffer	a direct ByteBuffer or a ERROR(/MappedByteBuffer) of the embedding model.

Throws

	if there is an error during AudioEmbedder creation.

public static AudioEmbedder createFromFile (Context context, String modelPath)

Creates an AudioEmbedder instance from a model file and default AudioEmbedder.AudioEmbedderOptions.

Parameters

context	an Android ERROR(/Context).
modelPath	path to the embedding model in the assets.

Throws

	if there is an error during AudioEmbedder creation.

public static AudioEmbedder createFromFile (Context context, File modelFile)

Creates an AudioEmbedder instance from a model file and default AudioEmbedder.AudioEmbedderOptions.

Parameters

context	an Android ERROR(/Context).
modelFile	the embedding model File instance.

Throws

IOException	if an I/O error occurs when opening the tflite model file.
	if there is an error during AudioEmbedder creation.

public static AudioEmbedder createFromOptions (Context context, AudioEmbedder.AudioEmbedderOptions options)

Creates an AudioEmbedder instance from an AudioEmbedder.AudioEmbedderOptions instance.

Parameters

context	an Android ERROR(/Context).
options	an AudioEmbedder.AudioEmbedderOptions instance.

Throws

	if there is an error during AudioEmbedder creation.

public AudioEmbedderResult embed (AudioData audioClip)

Parameters

audioClip

public void embedAsync (AudioData audioBlock, long timestampMs)

Parameters

audioBlock
timestampMs