HandLandmarker

public final class HandLandmarker

Performs hand landmarks detection on images.

This API expects a pre-trained hand landmarks model asset bundle. See .

Input image MPImage
- The image that hand landmarks detection runs on.
Output HandLandmarkerResult HandLandmarkerResult
- A HandLandmarkerResult containing hand landmarks.

Nested Classes

class HandLandmarker.HandLandmarkerOptions Options for setting up an HandLandmarker.

Fields

public static final Set<Connection>	HAND_CONNECTIONS
public static final Set<Connection>	HAND_INDEX_FINGER_CONNECTIONS
public static final Set<Connection>	HAND_MIDDLE_FINGER_CONNECTIONS
public static final Set<Connection>	HAND_PALM_CONNECTIONS
public static final Set<Connection>	HAND_PINKY_FINGER_CONNECTIONS
public static final Set<Connection>	HAND_RING_FINGER_CONNECTIONS
public static final Set<Connection>	HAND_THUMB_CONNECTIONS

Public Methods

static HandLandmarker	createFromBuffer(Context context, ByteBuffer modelBuffer) Creates a `HandLandmarker` instance from a model buffer and the default `HandLandmarker.HandLandmarkerOptions`.
static HandLandmarker	createFromFile(Context context, String modelPath) Creates a `HandLandmarker` instance from a model file and the default `HandLandmarker.HandLandmarkerOptions`.
static HandLandmarker	createFromFile(Context context, File modelFile) Creates a `HandLandmarker` instance from a model file and the default `HandLandmarker.HandLandmarkerOptions`.
static HandLandmarker	createFromOptions(Context context, HandLandmarker.HandLandmarkerOptions landmarkerOptions) Creates a `HandLandmarker` instance from a `HandLandmarker.HandLandmarkerOptions`.
HandLandmarkerResult	detect(MPImage image) Performs hand landmarks detection on the provided single image with default image processing options, i.e.
HandLandmarkerResult	detect(MPImage image, ImageProcessingOptions imageProcessingOptions) Performs hand landmarks detection on the provided single image.
void	detectAsync(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs) Sends live image data to perform hand landmarks detection, and the results will be available via the `OutputHandler.ResultListener` provided in the `HandLandmarker.HandLandmarkerOptions`.
void	detectAsync(MPImage image, long timestampMs) Sends live image data to perform hand landmarks detection with default image processing options, i.e.
HandLandmarkerResult	detectForVideo(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs) Performs hand landmarks detection on the provided video frame.
HandLandmarkerResult	detectForVideo(MPImage image, long timestampMs) Performs hand landmarks detection on the provided video frame with default image processing options, i.e.

Inherited Methods

From class com.google.mediapipe.tasks.vision.core.BaseVisionTaskApi

void	close() Closes and cleans up the MediaPipe vision task.
static long	generateResultTimestampMs(RunningMode runningMode, Packet packet) Generates the timestamp of a vision task result object from the vision task running mode and the output packet.

From class java.lang.Object

boolean	equals(Object arg0)
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

From interface java.lang.AutoCloseable

abstract void

close()

Fields

public static final Set<Connection> HAND_CONNECTIONS

public static final Set<Connection> HAND_INDEX_FINGER_CONNECTIONS

public static final Set<Connection> HAND_MIDDLE_FINGER_CONNECTIONS

public static final Set<Connection> HAND_PALM_CONNECTIONS

public static final Set<Connection> HAND_PINKY_FINGER_CONNECTIONS

public static final Set<Connection> HAND_RING_FINGER_CONNECTIONS

public static final Set<Connection> HAND_THUMB_CONNECTIONS

Public Methods

public static HandLandmarker createFromBuffer (Context context, ByteBuffer modelBuffer)

Creates a HandLandmarker instance from a model buffer and the default HandLandmarker.HandLandmarkerOptions.

Parameters

context	an Android `ERROR(/Context)`.
modelBuffer	a direct `ByteBuffer` or a `ERROR(/MappedByteBuffer)` of the detection model.

Throws

	if there is an error during `HandLandmarker` creation.

public static HandLandmarker createFromFile (Context context, String modelPath)

Creates a HandLandmarker instance from a model file and the default HandLandmarker.HandLandmarkerOptions.

Parameters

context	an Android `ERROR(/Context)`.
modelPath	path to the hand landmarks model with metadata in the assets.

Throws

	if there is an error during `HandLandmarker` creation.

public static HandLandmarker createFromFile (Context context, File modelFile)

Creates a HandLandmarker instance from a model file and the default HandLandmarker.HandLandmarkerOptions.

Parameters

context	an Android `ERROR(/Context)`.
modelFile	the hand landmarks model `File` instance.

Throws

IOException	if an I/O error occurs when opening the tflite model file.
	if there is an error during `HandLandmarker` creation.

public static HandLandmarker createFromOptions (Context context, HandLandmarker.HandLandmarkerOptions landmarkerOptions)

Creates a HandLandmarker instance from a HandLandmarker.HandLandmarkerOptions.

Parameters

context	an Android `ERROR(/Context)`.
landmarkerOptions	a `HandLandmarker.HandLandmarkerOptions` instance.

Throws

	if there is an error during `HandLandmarker` creation.

public HandLandmarkerResult detect (MPImage image)

Performs hand landmarks detection on the provided single image with default image processing options, i.e. without any rotation applied. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.IMAGE). TODO update java doc for input image format.

HandLandmarker supports the following color space types:

ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe `MPImage` object for processing.

Throws

	if there is an internal error.

public HandLandmarkerResult detect (MPImage image, ImageProcessingOptions imageProcessingOptions)

Performs hand landmarks detection on the provided single image. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.IMAGE). TODO update java doc for input image format.

HandLandmarker supports the following color space types:

ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe `MPImage` object for processing.
imageProcessingOptions	the `ImageProcessingOptions` specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying `ImageProcessingOptions.regionOfInterest()` will result in this method throwing an IllegalArgumentException.

Throws

IllegalArgumentException	if the `ImageProcessingOptions` specify a region-of-interest.
	if there is an internal error.

public void detectAsync (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)

Sends live image data to perform hand landmarks detection, and the results will be available via the OutputHandler.ResultListener provided in the HandLandmarker.HandLandmarkerOptions. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.LIVE_STREAM).

It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the hand landmarker. The input timestamps must be monotonically increasing.

HandLandmarker supports the following color space types:

ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe `MPImage` object for processing.
imageProcessingOptions	the `ImageProcessingOptions` specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying `ImageProcessingOptions.regionOfInterest()` will result in this method throwing an IllegalArgumentException.
timestampMs	the input timestamp (in milliseconds).

Throws

IllegalArgumentException	if the `ImageProcessingOptions` specify a region-of-interest.
	if there is an internal error.

public void detectAsync (MPImage image, long timestampMs)

Sends live image data to perform hand landmarks detection with default image processing options, i.e. without any rotation applied, and the results will be available via the OutputHandler.ResultListener provided in the HandLandmarker.HandLandmarkerOptions. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.LIVE_STREAM).

It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the hand landmarker. The input timestamps must be monotonically increasing.

HandLandmarker supports the following color space types:

ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe `MPImage` object for processing.
timestampMs	the input timestamp (in milliseconds).

Throws

	if there is an internal error.

public HandLandmarkerResult detectForVideo (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)

Performs hand landmarks detection on the provided video frame. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.VIDEO).

It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.

HandLandmarker supports the following color space types:

ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe `MPImage` object for processing.
imageProcessingOptions	the `ImageProcessingOptions` specifying how to process the input image before running inference. Note that region-of-interest is not supported by this task: specifying `ImageProcessingOptions.regionOfInterest()` will result in this method throwing an IllegalArgumentException.
timestampMs	the input timestamp (in milliseconds).

Throws

IllegalArgumentException	if the `ImageProcessingOptions` specify a region-of-interest.
	if there is an internal error.

public HandLandmarkerResult detectForVideo (MPImage image, long timestampMs)

Performs hand landmarks detection on the provided video frame with default image processing options, i.e. without any rotation applied. Only use this method when the HandLandmarker is created with ERROR(/RunningMode.VIDEO).

It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.

HandLandmarker supports the following color space types:

ERROR(/Bitmap.Config.ARGB_8888)

Parameters

image	a MediaPipe `MPImage` object for processing.
timestampMs	the input timestamp (in milliseconds).

Throws

	if there is an internal error.