Attention: This MediaPipe Solutions Preview is an early release. Learn more.

AudioClassifier

Stay organized with collections Save and categorize content based on your preferences.
public final class AudioClassifier

Performs audio classification on audio clips or audio stream.

This API expects a TFLite model with mandatory TFLite Model Metadata that contains the mandatory AudioProperties of the solo input audio tensor and the optional (but recommended) label items as AssociatedFiles with type TENSOR_AXIS_LABELS per output classification tensor.

Input tensor: (kTfLiteFloat32)

  • input audio buffer of size `[batch * samples]`.
  • batch inference is not supported (`batch` is required to be 1).
  • for multi-channel models, the channels need be interleaved.

At least one output tensor with: (kTfLiteFloat32)

  • `[1 x N]` array with `N` represents the number of categories.
  • optional (but recommended) label items as AssociatedFiles with type TENSOR_AXIS_LABELS, containing one label per line. The first such AssociatedFile (if any) is used to fill the `category_name` field of the results. The `display_name` field is filled from the AssociatedFile (if any) whose locale matches the `display_names_locale` field of the `AudioClassifierOptions` used at creation time ("en" by default, i.e. English). If none of these are available, only the `index` field of the results will be filled.

Nested Classes

class AudioClassifier.AudioClassifierOptions Options for setting up and AudioClassifier

Public Methods

AudioClassifierResult
classify(AudioData audioClip)
void
classifyAsync(AudioData audioBlock, long timestampMs)
static AudioClassifier
createFromBuffer(Context context, ByteBuffer modelBuffer)
Creates an AudioClassifier instance from a model buffer and default AudioClassifier.AudioClassifierOptions.
static AudioClassifier
createFromFile(Context context, String modelPath)
Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.
static AudioClassifier
createFromFile(Context context, File modelFile)
Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.
static AudioClassifier

Inherited Methods

Public Methods

public AudioClassifierResult classify (AudioData audioClip)

Parameters
audioClip

public void classifyAsync (AudioData audioBlock, long timestampMs)

Parameters
audioBlock
timestampMs

public static AudioClassifier createFromBuffer (Context context, ByteBuffer modelBuffer)

Creates an AudioClassifier instance from a model buffer and default AudioClassifier.AudioClassifierOptions.

Parameters
context an Android ERROR(/Context).
modelBuffer a direct ByteBuffer or a ERROR(/MappedByteBuffer) of the classification model.
Throws
if there is an error during AudioClassifier creation.

public static AudioClassifier createFromFile (Context context, String modelPath)

Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.

Parameters
context an Android ERROR(/Context).
modelPath path to the classification model in the assets.
Throws
if there is an error during AudioClassifier creation.

public static AudioClassifier createFromFile (Context context, File modelFile)

Creates an AudioClassifier instance from a model file and default AudioClassifier.AudioClassifierOptions.

Parameters
context an Android ERROR(/Context).
modelFile the classification model File instance.
Throws
IOException if an I/O error occurs when opening the tflite model file.
if there is an error during AudioClassifier creation.

public static AudioClassifier createFromOptions (Context context, AudioClassifier.AudioClassifierOptions options)

Creates an AudioClassifier instance from an AudioClassifier.AudioClassifierOptions instance.

Parameters
context an Android ERROR(/Context).
options an AudioClassifier.AudioClassifierOptions instance.
Throws
if there is an error during AudioClassifier creation.