MediaPipeTasksRetrieval Framework Reference

AudioData

class AudioData: NSObject

A wrapper class for input audio samples used in on-device machine learning using MediaPipe Task library.

Internally this class uses a ring buffer to hold input audio data. Clients could feed input audio data via load method and access the aggregated audio samples via buffer property.

Note that this class can only handle input audio in Float. TODO: Investigate if AVAudioFormatInt32 should be supported. Only necessary if you can change the input format of AVAudioEngine or load audio files using AVAudioFormatInt32.

• init(format:sampleCount:)

  Initializes a new instance of AudioData with the given AudioFormat and sample count.

  The AudioData stores data in a ring buffer of length sampleCount * MPPAudioFormat.channelCount.

  Declaration

  Swift

  init(format: AudioDataFormat, sampleCount: UInt)

  Parameters

  format	An instance of AudioFormat that defines the number of channels and sample rate required by the TFLite model.
  sampleCount	The number of samples AudioData can store at any given time. The sampleCount provided will be used to calculate the buffer length of AudioData (with bufferLength = format.channelCount * sampleCount.

  Return Value

  A new instance of AudioData with the given audio format and sample count.

• buffer

  A copy of all the internal buffer elements in order with the most recent elements appearing at the end of the buffer.

  Declaration

  Swift

  var buffer: MPPFloatBuffer { get }

• bufferLength

  Capacity of the AudioData buffer in number of elements.

  Declaration

  Swift

  var bufferLength: UInt { get }

• format

  Audio format specifying the number of channels and sample rate supported.

  Declaration

  Swift

  var format: AudioDataFormat { get }

• load(audioRecord:)

  Loads the internal buffer of AudioData with the audio samples currently held by the audio record.

  New data from the audio record is appended to the end of the buffer by shifting out any old data from the beginning of the buffer if needed to make space. If the length of the new data to be copied is more than the capacity of the buffer, only the most recent data of the AudioData's buffer will be copied from the input buffer.

  Declaration

  Swift

  func load(audioRecord: AudioRecord) throws

  Parameters

  audioRecord

  MPPAudioRecord whose values are to be loaded into AudioData. For multi-channel input, the audio record must hold interleaved data.

  Return Value

  A boolean indicating if the load operation succeeded.

• load(buffer:offset:length:)

  Loads the internal buffer of AudioData with a slice of the audio samples contained in the provided FloatBuffer.

  New data from the input buffer is appended to the end of the buffer by shifting out any old data from the beginning of the buffer if needed to make space. If the length of the new data to be copied is more than the capacity of the buffer, only the most recent data of the AudioData's buffer will be copied from the input buffer.

  Declaration

  Swift

  func load(buffer: MPPFloatBuffer, offset: UInt, length: UInt) throws

  Parameters

  sourceBuffer	A float buffer whose values are to be loaded into AudioData. For multi-channel input, the array must be interleaved.
  offset	Starting index in the source buffer from which elements should be copied.
  length	The number of elements to be copied.

  Return Value

  A boolean indicating if the load operation succeeded.