The MediaPipe Face Landmarker task lets you detect face landmarks and facial expressions in
images and videos. You can use this task to identify human facial expressions,
apply facial filters and effects, and create virtual avatars. This task uses
machine learning (ML) models that can work with single images or a continuous
stream of images. The task outputs 3-dimensional face landmarks, blendshape
scores (coefficients representing facial expression) to infer detailed facial
surfaces in real time, and transformation matrices to perform the
transformations required for effects rendering.
[Open in Web Demos](https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/face_landmarker)[Edit in StackBlitz](https://stackblitz.com/github/google-ai-edge/mediapipe-samples-web?file=src/tasks/face-landmarker.ts)
## Quickstart
Install the SDK for your platform,
[prepare input data](https://developers.google.com/edge/mediapipe/solutions/prepare_data.md.txt#vision), choose a
running mode, and run inference.
### Web
npm`npm install @mediapipe/tasks-vision`[Web demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/face_landmarker)·[Edit in StackBlitz](https://stackblitz.com/github/google-ai-edge/mediapipe-samples-web?file=src/tasks/face-landmarker.ts)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_web.md.txt)
#### Image
`IMAGE`Stateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
```typescript
import { FilesetResolver, FaceLandmarker } from "@mediapipe/tasks-vision";
const vision = await FilesetResolver.forVisionTasks(
"https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
);
const faceLandmarker = await FaceLandmarker.createFromOptions(vision, {
baseOptions: { modelAssetPath: "face_landmarker.task", delegate: "GPU" },
runningMode: "IMAGE",
numFaces: 1,
outputFaceBlendshapes: true,
});
// Synchronous single-frame call (stateless across images)
const image = document.getElementById("image") as HTMLImageElement;
const result = faceLandmarker.detect(image);
```
#### Video
`VIDEO`Every frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
```typescript
import { FilesetResolver, FaceLandmarker } from "@mediapipe/tasks-vision";
const vision = await FilesetResolver.forVisionTasks(
"https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
);
const faceLandmarker = await FaceLandmarker.createFromOptions(vision, {
baseOptions: { modelAssetPath: "face_landmarker.task", delegate: "GPU" },
runningMode: "VIDEO",
numFaces: 1,
outputFaceBlendshapes: true,
});
// Synchronous across decoded frames with monotonic timestamps (never drops frames)
for (const { frame, timestampMs } of decodedVideoFrames) {
const result = faceLandmarker.detectForVideo(frame, timestampMs);
}
```
#### Livestream
`VIDEO + rAF`Live camera · latest frameWeb tasks run synchronously---pair `runningMode: "VIDEO"` with `requestAnimationFrame` so each paint inspects only the latest camera frame (skipping intermediate frames when busy).
```typescript
import { FilesetResolver, FaceLandmarker } from "@mediapipe/tasks-vision";
const vision = await FilesetResolver.forVisionTasks(
"https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
);
// On Web, live camera loops pair runningMode: "VIDEO" with requestAnimationFrame
const faceLandmarker = await FaceLandmarker.createFromOptions(vision, {
baseOptions: { modelAssetPath: "face_landmarker.task", delegate: "GPU" },
runningMode: "VIDEO",
numFaces: 1,
outputFaceBlendshapes: true,
});
const video = document.getElementById("webcam") as HTMLVideoElement;
video.srcObject = await navigator.mediaDevices.getUserMedia({ video: true });
await video.play();
let lastVideoTime = -1;
function renderLoop() {
if (video.readyState >= 2 && video.currentTime !== lastVideoTime) {
lastVideoTime = video.currentTime;
const result = faceLandmarker.detectForVideo(video, performance.now());
console.log(result.faceLandmarks, result.faceBlendshapes);
}
requestAnimationFrame(renderLoop);
}
renderLoop();
```
### Android (Kotlin)
Gradle`implementation("com.google.mediapipe:tasks-vision:latest.release")`[Sample code](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/face_landmarker/android)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_android.md.txt)
#### Image
`IMAGE`Stateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
```kotlin
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.framework.image.MPImage
import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.facelandmarker.FaceLandmarker
val baseOptions = BaseOptions.builder()
.setModelAssetPath("face_landmarker.task")
.build()
val options = FaceLandmarker.FaceLandmarkerOptions.builder()
.setBaseOptions(baseOptions)
.setRunningMode(RunningMode.IMAGE)
.setNumFaces(1)
.setOutputFaceBlendshapes(true)
.build()
val faceLandmarker = FaceLandmarker.createFromOptions(context, options)
// Synchronous single-image call (no temporal tracking across calls)
val mpImage = BitmapImageBuilder(bitmap).build()
val result = faceLandmarker.detect(mpImage)
```
#### Video
`VIDEO`Every frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
```kotlin
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.framework.image.MPImage
import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.facelandmarker.FaceLandmarker
val baseOptions = BaseOptions.builder()
.setModelAssetPath("face_landmarker.task")
.build()
val options = FaceLandmarker.FaceLandmarkerOptions.builder()
.setBaseOptions(baseOptions)
.setRunningMode(RunningMode.VIDEO)
.setNumFaces(1)
.setOutputFaceBlendshapes(true)
.build()
val faceLandmarker = FaceLandmarker.createFromOptions(context, options)
// Synchronous: processes every frame in order with monotonic timestamps (no frame drops)
val mpImage = BitmapImageBuilder(frameBitmap).build()
val result = faceLandmarker.detectForVideo(mpImage, frameTimestampMs)
```
#### Livestream
`LIVE_STREAM`Async · drops frames when busyAsynchronous camera pipeline with a result callback. Automatically drops incoming frames when busy so results stay current with low latency.
```kotlin
import android.os.SystemClock
import android.util.Log
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.framework.image.MPImage
import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.facelandmarker.FaceLandmarker
val baseOptions = BaseOptions.builder()
.setModelAssetPath("face_landmarker.task")
.build()
val options = FaceLandmarker.FaceLandmarkerOptions.builder()
.setBaseOptions(baseOptions)
.setRunningMode(RunningMode.LIVE_STREAM)
.setNumFaces(1)
.setOutputFaceBlendshapes(true)
.setResultListener { result, _ -> val faces = result.faceLandmarks() }
.setErrorListener { error -> Log.e("MediaPipe", error.message.orEmpty()) }
.build()
val faceLandmarker = FaceLandmarker.createFromOptions(context, options)
// Async: automatically drops incoming camera frames when busy so results stay current
faceLandmarker.detectAsync(mpImage, SystemClock.uptimeMillis())
```
### Android (Java)
Gradle`implementation 'com.google.mediapipe:tasks-vision:latest.release'`[Sample code](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/face_landmarker/android)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_android.md.txt)
#### Image
`IMAGE`Stateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
```java
import com.google.mediapipe.framework.image.BitmapImageBuilder;
import com.google.mediapipe.framework.image.MPImage;
import com.google.mediapipe.tasks.core.BaseOptions;
import com.google.mediapipe.tasks.vision.core.RunningMode;
import com.google.mediapipe.tasks.vision.facelandmarker.FaceLandmarker;
import com.google.mediapipe.tasks.vision.facelandmarker.FaceLandmarkerResult;
BaseOptions baseOptions = BaseOptions.builder()
.setModelAssetPath("face_landmarker.task")
.build();
FaceLandmarker.FaceLandmarkerOptions options = FaceLandmarker.FaceLandmarkerOptions.builder()
.setBaseOptions(baseOptions)
.setRunningMode(RunningMode.IMAGE)
.setNumFaces(1)
.setOutputFaceBlendshapes(true)
.build();
FaceLandmarker faceLandmarker = FaceLandmarker.createFromOptions(context, options);
// Synchronous single-image call (no temporal tracking across calls)
MPImage mpImage = new BitmapImageBuilder(bitmap).build();
FaceLandmarkerResult result = faceLandmarker.detect(mpImage);
```
#### Video
`VIDEO`Every frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
```java
import com.google.mediapipe.framework.image.BitmapImageBuilder;
import com.google.mediapipe.framework.image.MPImage;
import com.google.mediapipe.tasks.core.BaseOptions;
import com.google.mediapipe.tasks.vision.core.RunningMode;
import com.google.mediapipe.tasks.vision.facelandmarker.FaceLandmarker;
import com.google.mediapipe.tasks.vision.facelandmarker.FaceLandmarkerResult;
BaseOptions baseOptions = BaseOptions.builder()
.setModelAssetPath("face_landmarker.task")
.build();
FaceLandmarker.FaceLandmarkerOptions options = FaceLandmarker.FaceLandmarkerOptions.builder()
.setBaseOptions(baseOptions)
.setRunningMode(RunningMode.VIDEO)
.setNumFaces(1)
.setOutputFaceBlendshapes(true)
.build();
FaceLandmarker faceLandmarker = FaceLandmarker.createFromOptions(context, options);
// Synchronous: processes every frame in order with monotonic timestamps (no frame drops)
MPImage mpImage = new BitmapImageBuilder(frameBitmap).build();
FaceLandmarkerResult result = faceLandmarker.detectForVideo(mpImage, frameTimestampMs);
```
#### Livestream
`LIVE_STREAM`Async · drops frames when busyAsynchronous camera pipeline with a result callback. Automatically drops incoming frames when busy so results stay current with low latency.
```java
import android.os.SystemClock;
import android.util.Log;
import com.google.mediapipe.framework.image.BitmapImageBuilder;
import com.google.mediapipe.framework.image.MPImage;
import com.google.mediapipe.tasks.core.BaseOptions;
import com.google.mediapipe.tasks.vision.core.RunningMode;
import com.google.mediapipe.tasks.vision.facelandmarker.FaceLandmarker;
BaseOptions baseOptions = BaseOptions.builder()
.setModelAssetPath("face_landmarker.task")
.build();
FaceLandmarker.FaceLandmarkerOptions options = FaceLandmarker.FaceLandmarkerOptions.builder()
.setBaseOptions(baseOptions)
.setRunningMode(RunningMode.LIVE_STREAM)
.setNumFaces(1)
.setOutputFaceBlendshapes(true)
.setResultListener((result, inputImage) -> { var faces = result.faceLandmarks(); })
.setErrorListener(error -> Log.e("MediaPipe", String.valueOf(error.getMessage())))
.build();
FaceLandmarker faceLandmarker = FaceLandmarker.createFromOptions(context, options);
// Async: automatically drops incoming camera frames when busy so results stay current
faceLandmarker.detectAsync(mpImage, SystemClock.uptimeMillis());
```
### iOS
SwiftPM`https://github.com/google-ai-edge/mediapipe → MediaPipeTasksVision`[Package.swift](https://developers.google.com/edge/mediapipe/solutions/setup_ios.md.txt#package_swift)·[Sample code](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/face_landmarker/ios)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_ios.md.txt)
#### Image
`IMAGE`Stateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
```swift
import Foundation
import MediaPipeTasksVision
guard let modelPath = Bundle.main.path(forResource: "face_landmarker", ofType: "task") else { return }
let options = FaceLandmarkerOptions()
options.baseOptions.modelAssetPath = modelPath
options.runningMode = .image
options.numFaces = 1
options.outputFaceBlendshapes = true
let faceLandmarker = try FaceLandmarker(options: options)
// Synchronous single-image call (stateless across images)
let mpImage = try MPImage(uiImage: image)
let result = try faceLandmarker.detect(image: mpImage)
```
#### Video
`VIDEO`Every frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
```swift
import Foundation
import MediaPipeTasksVision
guard let modelPath = Bundle.main.path(forResource: "face_landmarker", ofType: "task") else { return }
let options = FaceLandmarkerOptions()
options.baseOptions.modelAssetPath = modelPath
options.runningMode = .video
options.numFaces = 1
options.outputFaceBlendshapes = true
let faceLandmarker = try FaceLandmarker(options: options)
// Synchronous: processes every frame in order with monotonic timestamps (no frame drops)
let mpImage = try MPImage(pixelBuffer: pixelBuffer)
let result = try faceLandmarker.detect(videoFrame: mpImage, timestampInMilliseconds: timestampMs)
```
#### Livestream
`LIVE_STREAM`Async · drops frames when busyAsynchronous camera pipeline with a result callback. Automatically drops incoming frames when busy so results stay current with low latency.
```swift
import Foundation
import MediaPipeTasksVision
final class FaceLandmarkerStreamService: NSObject {
private var faceLandmarker: FaceLandmarker?
func setUp() throws {
guard let modelPath = Bundle.main.path(forResource: "face_landmarker", ofType: "task") else { return }
let options = FaceLandmarkerOptions()
options.baseOptions.modelAssetPath = modelPath
options.runningMode = .liveStream
options.numFaces = 1
options.outputFaceBlendshapes = true
options.faceLandmarkerLiveStreamDelegate = self
faceLandmarker = try FaceLandmarker(options: options)
}
func detectAsync(mpImage: MPImage, timestampMs: Int) throws {
// Async: automatically drops incoming camera frames when busy so results stay current
try faceLandmarker?.detectAsync(image: mpImage, timestampInMilliseconds: timestampMs)
}
}
extension FaceLandmarkerStreamService: FaceLandmarkerLiveStreamDelegate {
func faceLandmarker(
_ faceLandmarker: FaceLandmarker,
didFinishDetection result: FaceLandmarkerResult?,
timestampInMilliseconds: Int,
error: (any Error)?
) {
guard let result else { return }
DispatchQueue.main.async {
let faces = result.faceLandmarks
}
}
}
```
### Python
pip`pip install mediapipe`[Sample code](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/face_landmarker/python/%5BMediaPipe_Python_Tasks%5D_Face_Landmarker.ipynb)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_python.md.txt)
#### Image
`IMAGE`Stateless · single frameSynchronous call on independent still images. Resets temporal tracking state on every call.
```python
import mediapipe as mp
BaseOptions = mp.tasks.BaseOptions
FaceLandmarker = mp.tasks.vision.FaceLandmarker
FaceLandmarkerOptions = mp.tasks.vision.FaceLandmarkerOptions
VisionRunningMode = mp.tasks.vision.RunningMode
options = FaceLandmarkerOptions(
base_options=BaseOptions(model_asset_path="face_landmarker.task"),
running_mode=VisionRunningMode.IMAGE,
num_faces=1,
output_face_blendshapes=True,
)
with FaceLandmarker.create_from_options(options) as landmarker:
# Synchronous single-image call (stateless across images)
mp_image = mp.Image.create_from_file("input.jpg")
result = landmarker.detect(mp_image)
```
#### Video
`VIDEO`Every frame · no dropsSynchronous tracking across decoded video frames with monotonic timestamps. Processes every frame in order and never drops frames.
```python
import mediapipe as mp
BaseOptions = mp.tasks.BaseOptions
FaceLandmarker = mp.tasks.vision.FaceLandmarker
FaceLandmarkerOptions = mp.tasks.vision.FaceLandmarkerOptions
VisionRunningMode = mp.tasks.vision.RunningMode
options = FaceLandmarkerOptions(
base_options=BaseOptions(model_asset_path="face_landmarker.task"),
running_mode=VisionRunningMode.VIDEO,
num_faces=1,
output_face_blendshapes=True,
)
with FaceLandmarker.create_from_options(options) as landmarker:
# Synchronous: processes every frame in order with monotonic timestamps (no frame drops)
for mp_image, timestamp_ms in decoded_video_frames:
result = landmarker.detect_for_video(mp_image, timestamp_ms)
```
#### Livestream
`LIVE_STREAM`Async · drops frames when busyAsynchronous camera pipeline with a result callback. Automatically drops incoming frames when busy so results stay current with low latency.
```python
import mediapipe as mp
BaseOptions = mp.tasks.BaseOptions
FaceLandmarker = mp.tasks.vision.FaceLandmarker
FaceLandmarkerOptions = mp.tasks.vision.FaceLandmarkerOptions
VisionRunningMode = mp.tasks.vision.RunningMode
def on_result(result, output_image: mp.Image, timestamp_ms: int):
print(f"Detected {len(result.face_landmarks)} face(s) at {timestamp_ms} ms")
options = FaceLandmarkerOptions(
base_options=BaseOptions(model_asset_path="face_landmarker.task"),
running_mode=VisionRunningMode.LIVE_STREAM,
num_faces=1,
output_face_blendshapes=True,
result_callback=on_result,
)
with FaceLandmarker.create_from_options(options) as landmarker:
# Async: automatically drops incoming camera frames when busy so results stay current
mp_image = mp.Image(image_format=mp.ImageFormat.SRGB, data=rgb_frame)
landmarker.detect_async(mp_image, timestamp_ms)
```
### Flutter
pub`flutter pub add mediapipe_vision`[Live gallery](https://google.github.io/flutter-mediapipe/)·[Sample code](https://github.com/google/flutter-mediapipe/tree/main/samples/vision/face_landmarker)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_flutter.md.txt)
#### Image
`RunningMode.image`Stateless · single frameOne call per independent still image. No tracking state carries over between calls.
```dart
import 'package:mediapipe_vision/mediapipe_vision.dart';
Future readExpression(VisionImage image) async {
final landmarker = await FaceLandmarker.create(
FaceLandmarkerOptions(
model: VisionModels.faceLandmarker,
outputFaceBlendshapes: true,
),
);
try {
final result = await landmarker.detect(image);
if (result.faceBlendshapes.isEmpty) return;
final strongest = [...result.faceBlendshapes.first]
..sort((a, b) => b.score.compareTo(a.score));
for (final shape in strongest.take(3)) {
print('${shape.categoryName}: ${shape.score.toStringAsFixed(2)}');
}
} finally {
await landmarker.dispose();
}
}
```
#### Video
`RunningMode.video`Every frame · no dropsTracking across decoded video frames with strictly increasing timestamps. Processes every frame in order and never drops frames.
```dart
import 'dart:typed_data';
import 'package:mediapipe_vision/mediapipe_vision.dart';
typedef Frame = ({Uint8List rgba, int width, int height, int milliseconds});
Future readExpressionsInVideo(Stream frames) async {
final landmarker = await FaceLandmarker.create(
FaceLandmarkerOptions(
model: VisionModels.faceLandmarker,
runningMode: RunningMode.video,
outputFaceBlendshapes: true,
),
);
try {
await for (final frame in frames) {
final result = await landmarker.detectForVideo(
VisionImage.fromPixels(
pixels: frame.rgba,
width: frame.width,
height: frame.height,
format: VisionPixelFormat.rgba,
),
timestampMilliseconds: frame.milliseconds,
);
final ms = result.timestampMilliseconds;
for (final shapes in result.faceBlendshapes) {
final top = shapes.reduce((a, b) => a.score >= b.score ? a : b);
print('$ms ms: ${top.categoryName} ${top.score.toStringAsFixed(2)}');
}
}
} finally {
await landmarker.dispose();
}
}
```
#### Livestream
`RunningMode.liveStream`Async · drops frames when busyEach call returns at once and results arrive on the task's `results` stream. One frame runs at a time; a newer frame replaces one that's waiting, so results stay current.
```dart
import 'dart:typed_data';
import 'package:mediapipe_vision/mediapipe_vision.dart';
typedef Frame = ({Uint8List rgba, int width, int height, int milliseconds});
Future readExpressionsLive(Stream frames) async {
final landmarker = await FaceLandmarker.create(
FaceLandmarkerOptions(
model: VisionModels.faceLandmarker,
runningMode: RunningMode.liveStream,
outputFaceBlendshapes: true,
),
);
// Listen before the first frame, as Google requires a result listener.
final results = landmarker.results.listen((result) {
final ms = result.timestampMilliseconds;
for (final shapes in result.faceBlendshapes) {
final top = shapes.reduce((a, b) => a.score >= b.score ? a : b);
print('$ms ms: ${top.categoryName} ${top.score.toStringAsFixed(2)}');
}
});
try {
await for (final frame in frames) {
landmarker.detectAsync(
VisionImage.fromPixels(
pixels: frame.rgba,
width: frame.width,
height: frame.height,
format: VisionPixelFormat.rgba,
),
timestampMilliseconds: frame.milliseconds,
);
}
} finally {
// Runs the frame in flight and the waiting one, then closes results.
await landmarker.dispose();
await results.cancel();
print('${landmarker.droppedFrames} frames dropped');
}
}
```
On Web, you can use `DrawingUtils(ctx).drawConnectors(landmarks, FaceLandmarker.FACE_LANDMARKS_TESSELATION)` (along with `FACE_LANDMARKS_RIGHT_EYE`, `FACE_LANDMARKS_LEFT_EYE`, and `FACE_LANDMARKS_FACE_OVAL`) from `@mediapipe/tasks-vision` to render facial mesh overlays on an HTML `