AudioClassifier

public final class AudioClassifier

Performs audio classification on audio clips or audio stream.

This API expects a TFLite model with mandatory TFLite Model Metadata that contains the mandatory AudioProperties of the solo input audio tensor and the optional (but recommended) label items as AssociatedFiles with type TENSOR_AXIS_LABELS per output classification tensor.

Input tensor: (kTfLiteFloat32)

• input audio buffer of size `[batch * samples]`.
• batch inference is not supported (`batch` is required to be 1).
• for multi-channel models, the channels need be interleaved.

At least one output tensor with: (kTfLiteFloat32)

• `[1 x N]` array with `N` represents the number of categories.
• optional (but recommended) label items as AssociatedFiles with type TENSOR_AXIS_LABELS, containing one label per line. The first such AssociatedFile (if any) is used to fill the `category_name` field of the results. The `display_name` field is filled from the AssociatedFile (if any) whose locale matches the `display_names_locale` field of the `AudioClassifierOptions` used at creation time ("en" by default, i.e. English). If none of these are available, only the `index` field of the results will be filled.

Nested Classes

class

AudioClassifier.AudioClassifierOptions

Options for setting up and AudioClassifier.

Public Methods

AudioClassifierResult	classify(AudioData audioClip)
void	classifyAsync(AudioData audioBlock, long timestampMs)
static AudioClassifier	createFromBuffer(Context context, ByteBuffer modelBuffer) Creates an AudioClassifier instance from a model buffer and default AudioClassifier.AudioClassifierOptions.
static AudioClassifier	createFromFile(Context context, String modelPath) Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.
static AudioClassifier	createFromFile(Context context, File modelFile) Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.
static AudioClassifier	createFromOptions(Context context, AudioClassifier.AudioClassifierOptions options) Creates an AudioClassifier instance from an AudioClassifier.AudioClassifierOptions instance.

Inherited Methods

From class com.google.mediapipe.tasks.audio.core.BaseAudioTaskApi

void	close() Closes and cleans up the MediaPipe audio task.
AudioRecord	createAudioRecord() Creates an AudioRecord instance to record audio stream that has mono channel at sample rate at sample rate 16kHz, the sample rate required for models like Yamnet.
AudioRecord	createAudioRecord(int numChannels, int sampleRate, int requiredInputBufferSize) Creates an AudioRecord instance to record audio stream.

From class java.lang.Object

boolean	equals(Object arg0)
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

From interface java.lang.AutoCloseable

abstract void

close()

Public Methods

public AudioClassifierResult classify (AudioData audioClip)

Parameters

audioClip

public void classifyAsync (AudioData audioBlock, long timestampMs)

Parameters

audioBlock
timestampMs

public static AudioClassifier createFromBuffer (Context context, ByteBuffer modelBuffer)

Creates an AudioClassifier instance from a model buffer and default AudioClassifier.AudioClassifierOptions.

Parameters

context	an Android ERROR(/Context).
modelBuffer	a direct ByteBuffer or a ERROR(/MappedByteBuffer) of the classification model.

Throws

	if there is an error during AudioClassifier creation.

public static AudioClassifier createFromFile (Context context, String modelPath)

Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.

Parameters

context	an Android ERROR(/Context).
modelPath	path to the classification model in the assets.

Throws

	if there is an error during AudioClassifier creation.

public static AudioClassifier createFromFile (Context context, File modelFile)

Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.

Parameters

context	an Android ERROR(/Context).
modelFile	the classification model File instance.

Throws

IOException	if an I/O error occurs when opening the tflite model file.
	if there is an error during AudioClassifier creation.

public static AudioClassifier createFromOptions (Context context, AudioClassifier.AudioClassifierOptions options)

Creates an AudioClassifier instance from an AudioClassifier.AudioClassifierOptions instance.

Parameters

context	an Android ERROR(/Context).
options	an AudioClassifier.AudioClassifierOptions instance.

Throws

	if there is an error during AudioClassifier creation.