The MediaPipe Holistic Landmarker task lets you combine components of the face, hand, and pose landmarkers to detect human body landmarks in images or video. This task outputs holistic landmarks in normalized image coordinates and 3D world coordinates.
The code sample described in these instructions is available on GitHub. For more information about the capabilities, models, and configuration options of this task, see the Overview.
Code example
The example code for Holistic Landmarker provides a complete implementation of this task in Python for your reference. This code helps you test this task and get started on building your own holistic landmarker. You can view and run the Holistic Landmarker example code.
Setup
This section describes key steps for setting up your development environment and code projects specifically to use Holistic Landmarker. For general information on setting up your development environment for using MediaPipe tasks, including platform version requirements, see the Setup guide for Python.
Packages
The MediaPipe Holistic Landmarker task requires the mediapipe PyPI package. You can install and import these dependencies with the following:
$ python -m pip install mediapipe
Imports
Import the following classes to access the Holistic Landmarker task functions:
import mediapipe as mp
from mediapipe.tasks import python
from mediapipe.tasks.python import vision
Model
The MediaPipe Holistic Landmarker task requires a trained model bundle that is compatible with this task. For more information on available trained models for Holistic Landmarker, see the task overview Models section.
Select and download the model, and then store it in a local directory:
model_path = '/absolute/path/to/holistic_landmarker.task'
Use the BaseOptions object model_asset_path parameter to specify the path of
the model to use.
Create the task
The MediaPipe Holistic Landmarker task uses the create_from_options function to
set up the task. The create_from_options function accepts values
for configuration options to handle. For more information, see
Configuration options.
The following code demonstrates how to build and configure this task.
Image
import mediapipe as mp BaseOptions = mp.tasks.BaseOptions HolisticLandmarker = mp.tasks.vision.HolisticLandmarker HolisticLandmarkerOptions = mp.tasks.vision.HolisticLandmarkerOptions VisionRunningMode = mp.tasks.vision.RunningMode options = HolisticLandmarkerOptions( base_options=BaseOptions(model_asset_path=model_path), running_mode=VisionRunningMode.IMAGE, min_face_detection_confidence=0.5, min_pose_detection_confidence=0.5, min_hand_landmarks_confidence=0.5) with HolisticLandmarker.create_from_options(options) as landmarker: # The landmarker is initialized. Use it here. # ...
Video
import mediapipe as mp BaseOptions = mp.tasks.BaseOptions HolisticLandmarker = mp.tasks.vision.HolisticLandmarker HolisticLandmarkerOptions = mp.tasks.vision.HolisticLandmarkerOptions VisionRunningMode = mp.tasks.vision.RunningMode options = HolisticLandmarkerOptions( base_options=BaseOptions(model_asset_path=model_path), running_mode=VisionRunningMode.VIDEO, min_face_detection_confidence=0.5, min_pose_detection_confidence=0.5, min_hand_landmarks_confidence=0.5) with HolisticLandmarker.create_from_options(options) as landmarker: # The landmarker is initialized. Use it here. # ...
Live stream
import mediapipe as mp BaseOptions = mp.tasks.BaseOptions HolisticLandmarker = mp.tasks.vision.HolisticLandmarker HolisticLandmarkerOptions = mp.tasks.vision.HolisticLandmarkerOptions HolisticLandmarkerResult = mp.tasks.vision.HolisticLandmarkerResult VisionRunningMode = mp.tasks.vision.RunningMode def print_result(result: HolisticLandmarkerResult, output_image: mp.Image, timestamp_ms: int): print('holistic landmarker result: {}'.format(result)) options = HolisticLandmarkerOptions( base_options=BaseOptions(model_asset_path=model_path), running_mode=VisionRunningMode.LIVE_STREAM, min_face_detection_confidence=0.5, min_pose_detection_confidence=0.5, min_hand_landmarks_confidence=0.5, result_callback=print_result) with HolisticLandmarker.create_from_options(options) as landmarker: # The landmarker is initialized. Use it here. # ...
Configuration options
This task has the following configuration options for Python applications:
| Option Name | Description | Value Range | Default Value |
|---|---|---|---|
running_mode |
Sets the running mode for the task. There are three
modes: IMAGE: The mode for single image inputs. VIDEO: The mode for decoded frames of a video. LIVE_STREAM: The mode for a livestream of input data, such as from a camera. |
{IMAGE, VIDEO, LIVE_STREAM} |
IMAGE |
min_face_detection_confidence |
The minimum confidence score for the face detection to be considered successful. | Float [0.0, 1.0] |
0.5 |
min_face_suppression_threshold |
The minimum non-maximum-suppression threshold for face detection to be considered overlapped. | Float [0.0, 1.0] |
0.3 |
min_face_landmarks_confidence |
The minimum confidence score of face presence score in the face landmarks detection. | Float [0.0, 1.0] |
0.5 |
min_pose_detection_confidence |
The minimum confidence score for the pose detection to be considered successful. | Float [0.0, 1.0] |
0.5 |
min_pose_suppression_threshold |
The minimum non-maximum-suppression threshold for pose detection to be considered overlapped. | Float [0.0, 1.0] |
0.3 |
min_pose_landmarks_confidence |
The minimum confidence score of pose presence score in the pose landmarks detection. | Float [0.0, 1.0] |
0.5 |
min_hand_landmarks_confidence |
The minimum confidence score of hand presence score in the hand landmarks detection. | Float [0.0, 1.0] |
0.5 |
output_face_blendshapes |
Whether to output face blendshapes classification. Face blendshapes are used for rendering the 3D face model. | Boolean |
false |
output_segmentation_mask |
Whether to output segmentation masks for the human pose. | Boolean |
false |
Prepare data
Holistic Landmarker can detect holistic landmarks in images in any format supported by the host platform. The task also handles data input preprocessing, including resizing, rotation and value normalization. To detect landmarks in videos, you can use the API to quickly process one frame at a time, using the timestamp of the frame to determine when the landmarks occur within the video.
Run the task
Use the detect, detect_for_video, or detect_async methods to run the
landmarker depending on the running mode.
# Load the input image
mp_image = mp.Image.create_from_file('path/to/image.jpg')
# Detect holistic landmarks
result = landmarker.detect(mp_image)