HandLandmarker

public final class HandLandmarker

Performs hand landmarks detection on images.

This API expects a pre-trained hand landmarks model asset bundle. See .

• Input image MPImage
  • The image that hand landmarks detection runs on.
• Output HandLandmarkerResult HandLandmarkerResult
  • A HandLandmarkerResult containing hand landmarks.

Nested Classes

class

HandLandmarker.HandLandmarkerOptions

Options for setting up an HandLandmarker.

Fields

public static final Set<Connection>	HAND_CONNECTIONS
public static final Set<Connection>	HAND_INDEX_FINGER_CONNECTIONS
public static final Set<Connection>	HAND_MIDDLE_FINGER_CONNECTIONS
public static final Set<Connection>	HAND_PALM_CONNECTIONS
public static final Set<Connection>	HAND_PINKY_FINGER_CONNECTIONS
public static final Set<Connection>	HAND_RING_FINGER_CONNECTIONS
public static final Set<Connection>	HAND_THUMB_CONNECTIONS

Public Methods

static HandLandmarker	createFromBuffer(Context context, ByteBuffer modelBuffer) Creates a HandLandmarker instance from a model buffer and the default HandLandmarker.HandLandmarkerOptions.
static HandLandmarker	createFromFile(Context context, String modelPath) Creates a HandLandmarker instance from a model file and the default HandLandmarker.HandLandmarkerOptions.
static HandLandmarker	createFromFile(Context context, File modelFile) Creates a HandLandmarker instance from a model file and the default HandLandmarker.HandLandmarkerOptions.
static HandLandmarker	createFromOptions(Context context, HandLandmarker.HandLandmarkerOptions landmarkerOptions) Creates a HandLandmarker instance from a HandLandmarker.HandLandmarkerOptions.
HandLandmarkerResult	detect(MPImage image) Performs hand landmarks detection on the provided single image with default image processing options, i.e.
HandLandmarkerResult	detect(MPImage image, ImageProcessingOptions imageProcessingOptions) Performs hand landmarks detection on the provided single image.
void	detectAsync(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs) Sends live image data to perform hand landmarks detection, and the results will be available via the OutputHandler.ResultListener provided in the HandLandmarker.HandLandmarkerOptions.
void	detectAsync(MPImage image, long timestampMs) Sends live image data to perform hand landmarks detection with default image processing options, i.e.
HandLandmarkerResult	detectForVideo(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs) Performs hand landmarks detection on the provided video frame.
HandLandmarkerResult	detectForVideo(MPImage image, long timestampMs) Performs hand landmarks detection on the provided video frame with default image processing options, i.e.

Inherited Methods

From class com.google.mediapipe.tasks.vision.core.BaseVisionTaskApi

void	close() Closes and cleans up the MediaPipe vision task.
static long	generateResultTimestampMs(RunningMode runningMode, Packet packet) Generates the timestamp of a vision task result object from the vision task running mode and the output packet.

From class java.lang.Object

boolean	equals(Object arg0)
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

From interface java.lang.AutoCloseable

abstract void

close()

Fields

public static final Set<Connection> HAND_CONNECTIONS

public static final Set<Connection> HAND_INDEX_FINGER_CONNECTIONS

public static final Set<Connection> HAND_MIDDLE_FINGER_CONNECTIONS

public static final Set<Connection> HAND_PALM_CONNECTIONS

public static final Set<Connection> HAND_PINKY_FINGER_CONNECTIONS

public static final Set<Connection> HAND_RING_FINGER_CONNECTIONS

public static final Set<Connection> HAND_THUMB_CONNECTIONS

Public Methods

public static HandLandmarker createFromBuffer (Context context, ByteBuffer modelBuffer)

Creates a HandLandmarker instance from a model buffer and the default HandLandmarker.HandLandmarkerOptions.

Parameters

context	an Android ERROR(/Context).
modelBuffer	a direct ByteBuffer or a ERROR(/MappedByteBuffer) of the detection model.

Throws

	if there is an error during HandLandmarker creation.

public static HandLandmarker createFromFile (Context context, String modelPath)

Creates a HandLandmarker instance from a model file and the default HandLandmarker.HandLandmarkerOptions.

Parameters

context	an Android ERROR(/Context).
modelPath	path to the hand landmarks model with metadata in the assets.

Throws

	if there is an error during HandLandmarker creation.

public static HandLandmarker createFromFile (Context context, File modelFile)

Creates a HandLandmarker instance from a model file and the default HandLandmarker.HandLandmarkerOptions.

Parameters

context	an Android ERROR(/Context).
modelFile	the hand landmarks model File instance.

Throws

IOException	if an I/O error occurs when opening the tflite model file.
	if there is an error during HandLandmarker creation.

public static HandLandmarker createFromOptions (Context context, HandLandmarker.HandLandmarkerOptions landmarkerOptions)

Creates a HandLandmarker instance from a HandLandmarker.HandLandmarkerOptions.

Parameters

context	an Android ERROR(/Context).
landmarkerOptions	a HandLandmarker.HandLandmarkerOptions instance.

Throws

	if there is an error during HandLandmarker creation.

public HandLandmarkerResult detect (MPImage image)

Performs hand landmarks detection on the provided single image with default image processing options, i.e. without any rotation applied. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.IMAGE). TODO update java doc for input image format.

HandLandmarker supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.

Throws

	if there is an internal error.

public HandLandmarkerResult detect (MPImage image, ImageProcessingOptions imageProcessingOptions)

Performs hand landmarks detection on the provided single image. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.IMAGE). TODO update java doc for input image format.

HandLandmarker supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.
imageProcessingOptions	the ImageProcessingOptions specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying ImageProcessingOptions.regionOfInterest() will result in this method throwing an IllegalArgumentException.

Throws

IllegalArgumentException	if the ImageProcessingOptions specify a region-of-interest.
	if there is an internal error.

public void detectAsync (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)

Sends live image data to perform hand landmarks detection, and the results will be available via the OutputHandler.ResultListener provided in the HandLandmarker.HandLandmarkerOptions. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.LIVE_STREAM).

It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the hand landmarker. The input timestamps must be monotonically increasing.

HandLandmarker supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.
imageProcessingOptions	the ImageProcessingOptions specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying ImageProcessingOptions.regionOfInterest() will result in this method throwing an IllegalArgumentException.
timestampMs	the input timestamp (in milliseconds).

Throws

IllegalArgumentException	if the ImageProcessingOptions specify a region-of-interest.
	if there is an internal error.

public void detectAsync (MPImage image, long timestampMs)

Sends live image data to perform hand landmarks detection with default image processing options, i.e. without any rotation applied, and the results will be available via the OutputHandler.ResultListener provided in the HandLandmarker.HandLandmarkerOptions. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.LIVE_STREAM).

It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the hand landmarker. The input timestamps must be monotonically increasing.

HandLandmarker supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.
timestampMs	the input timestamp (in milliseconds).

Throws

	if there is an internal error.

public HandLandmarkerResult detectForVideo (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)

Performs hand landmarks detection on the provided video frame. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.VIDEO).

It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.

HandLandmarker supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.
imageProcessingOptions	the ImageProcessingOptions specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying ImageProcessingOptions.regionOfInterest() will result in this method throwing an IllegalArgumentException.
timestampMs	the input timestamp (in milliseconds).

Throws

IllegalArgumentException	if the ImageProcessingOptions specify a region-of-interest.
	if there is an internal error.

public HandLandmarkerResult detectForVideo (MPImage image, long timestampMs)

Performs hand landmarks detection on the provided video frame with default image processing options, i.e. without any rotation applied. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.VIDEO).

It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.

HandLandmarker supports the following color space types:

• ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe MPImage object for processing.
timestampMs	the input timestamp (in milliseconds).

Throws

	if there is an internal error.