MediaPipeTasksVision Framework Reference

MPPFaceLandmarker

@interface MPPFaceLandmarker : NSObject

@brief Class that performs face landmark detection on images.

The API expects a TFLite model with mandatory TFLite Model Metadata.

• -initWithModelPath:error:

  Creates a new instance of FaceLandmarker from an absolute path to a TensorFlow Lite model file stored locally on the device and the default FaceLandmarker.

  Declaration

  Objective-C

  - (nullable instancetype)initWithModelPath:(nonnull NSString *)modelPath
                                       error:(NSError *_Nullable *_Nullable)error;

  Parameters

  modelPath

  An absolute path to a TensorFlow Lite model file stored locally on the device.

  Return Value

  A new instance of FaceLandmarker with the given model path. nil if there is an error in initializing the face landmaker.

• -initWithOptions:error:

  Creates a new instance of FaceLandmarker from the given FaceLandmarkerOptions.

  Declaration

  Objective-C

  - (nullable instancetype)initWithOptions:
                               (nonnull MPPFaceLandmarkerOptions *)options
                                     error:(NSError *_Nullable *_Nullable)error;

  Parameters

  options

  The options of type FaceLandmarkerOptions to use for configuring the FaceLandmarker.

  Return Value

  A new instance of FaceLandmarker with the given options. nil if there is an error in initializing the face landmaker.

• -detectInImage:error:

  Performs face landmark detection on the provided MPImage using the whole image as region of interest. Rotation will be applied according to the orientation property of the provided MPImage. Only use this method when the FaceLandmarker is created with .image.

  This method supports RGBA images. If your MPImage has a source type of .pixelBuffer or .sampleBuffer, the underlying pixel buffer must have one of the following pixel format types:

  1. kCVPixelFormatType_32BGRA
  2. kCVPixelFormatType_32RGBA

  If your MPImage has a source type of .image ensure that the color space is RGB with an Alpha channel.

  Declaration

  Objective-C

  - (nullable MPPFaceLandmarkerResult *)
      detectInImage:(nonnull MPPImage *)image
              error:(NSError *_Nullable *_Nullable)error;

  Parameters

  image

  The MPImage on which face landmark detection is to be performed.

  Return Value

  An FaceLandmarkerResult that contains a list of landmarks. nil if there is an error in initializing the face landmaker.

• -detectInVideoFrame:timestampInMilliseconds:error:

  Performs face landmark detection on the provided video frame of type MPImage using the whole image as region of interest. Rotation will be applied according to the orientation property of the provided MPImage. Only use this method when the FaceLandmarker is created with running mode .video.

  This method supports RGBA images. If your MPImage has a source type of .pixelBuffer or .sampleBuffer, the underlying pixel buffer must have one of the following pixel format types:

  1. kCVPixelFormatType_32BGRA
  2. kCVPixelFormatType_32RGBA

  If your MPImage has a source type of .image ensure that the color space is RGB with an Alpha channel.

  Declaration

  Objective-C

  - (nullable MPPFaceLandmarkerResult *)
           detectInVideoFrame:(nonnull MPPImage *)image
      timestampInMilliseconds:(NSInteger)timestampInMilliseconds
                        error:(NSError *_Nullable *_Nullable)error;

  Parameters

  image	The MPImage on which face landmark detection is to be performed.
  timestampInMilliseconds	The video frame’s timestamp (in milliseconds). The input timestamps must be monotonically increasing.

  Return Value

  An FaceLandmarkerResult that contains a list of landmarks. nil if there is an error in initializing the face landmaker.

• -detectAsyncInImage:timestampInMilliseconds:error:

  Sends live stream image data of type MPImage to perform face landmark detection using the whole image as region of interest. Rotation will be applied according to the orientation property of the provided MPImage. Only use this method when the FaceLandmarker is created with .liveStream.

  The object which needs to be continuously notified of the available results of face detection must confirm to FaceLandmarkerLiveStreamDelegate protocol and implement the faceLandmarker(_:didFinishDetectionWithResult:timestampInMilliseconds:error:) delegate method.

  It’s required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the face detector. The input timestamps must be monotonically increasing.

  This method supports RGBA images. If your MPImage has a source type of .pixelBuffer or .sampleBuffer, the underlying pixel buffer must have one of the following pixel format types:

  1. kCVPixelFormatType_32BGRA
  2. kCVPixelFormatType_32RGBA

  If the input MPImage has a source type of .image ensure that the color space is RGB with an Alpha channel.

  If this method is used for classifying live camera frames using AVFoundation, ensure that you request AVCaptureVideoDataOutput to output frames in kCMPixelFormat_32RGBA using its videoSettings property.

  Declaration

  Objective-C

  - (BOOL)detectAsyncInImage:(nonnull MPPImage *)image
      timestampInMilliseconds:(NSInteger)timestampInMilliseconds
                        error:(NSError *_Nullable *_Nullable)error;

  Parameters

  image	A live stream image data of type MPImage on which face landmark detection is to be performed.
  timestampInMilliseconds	The timestamp (in milliseconds) which indicates when the input image is sent to the face detector. The input timestamps must be monotonically increasing.

  Return Value

  true if the image was sent to the task successfully, otherwise false.

• +lipsConnections

  Returns the connections between all the landmarks in the lips.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)lipsConnections;

  Return Value

  An array of connections between all the landmarks in the lips.

• +leftEyeConnections

  Returns the connections between all the landmarks in the left eye.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)leftEyeConnections;

  Return Value

  An array of connections between all the landmarks in the left eye.

• +leftEyebrowConnections

  Returns the connections between all the landmarks in the left eyebrow.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)leftEyebrowConnections;

  Return Value

  An array of connections between all the landmarks in the left eyebrow.

• +leftIrisConnections

  Returns the connections between all the landmarks in the left iris.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)leftIrisConnections;

  Return Value

  An array of connections between all the landmarks in the left iris.

• +rightEyeConnections

  Returns the connections between all the landmarks in the right eye.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)rightEyeConnections;

  Return Value

  An array of connections between all the landmarks in the right eyr.

• +rightEyebrowConnections

  Returns the connections between all the landmarks in the right eyebrow.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)rightEyebrowConnections;

  Return Value

  An array of connections between all the landmarks in the right eyebrow.

• +rightIrisConnections

  Returns the connections between all the landmarks in the right iris.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)rightIrisConnections;

  Return Value

  An array of connections between all the landmarks in the right iris.

• +faceOvalConnections

  Returns the connections between all the landmarks of the face oval.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)faceOvalConnections;

  Return Value

  An array of connections between all the landmarks of the face oval.

• +contoursConnections

  Returns the connections between making up the contours of the face.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)contoursConnections;

  Return Value

  An array of connections between all the contours of the face.

• +tesselationConnections

  Returns the connections between all the landmarks making up the tesselation of the face.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)tesselationConnections;

  Return Value

  An array of connections between all the landmarks making up the tesselation of the face.

• +faceConnections

  Returns the connections between all the landmarks in the face.

  Declaration

  Objective-C

  + (nonnull NSArray<MPPConnection *> *)faceConnections;

  Return Value

  An array of connections between all the landmarks in the face.

• -init

  Undocumented

  Declaration

  Objective-C

  - (instancetype)init NS_UNAVAILABLE;

• +new

  Undocumented

  Declaration

  Objective-C

  + (instancetype)new NS_UNAVAILABLE;