Learn how to use the Augmented Faces feature in your own apps.
Using Augmented Faces in Android
Configure the ARCore session
Initialize the session with FRONT_CAMERA. Note that selecting the front camera
will cause a number of changes
in ARCore behavior.
session = new Session(this, EnumSet.of(Session.Feature.FRONT_CAMERA))
Enable AugmentedFaceMode:
Config config = new Config(session);
config.setAugmentedFaceMode(AugmentedFaceMode.MESH3D);
session.configure(config);
Face mesh orientation
Note the orientation of the face mesh:
Access the detected face
Get a Trackable
for each frame. A Trackable is something that ARCore can track and that
Anchors can be attached to.
// ARCore's face detection works best on upright faces, relative to gravity.
Collection<AugmentedFace> faces = session.getAllTrackables(AugmentedFace.class);
Get the TrackingState
for each Trackable. TrackingState.TRACKING gets all of the information
stored in any given Trackable.
for (AugmentedFace face : faces) {
if (face.getTrackingState() == TrackingState.TRACKING) {
// UVs and indices can be cached as they do not change during the session.
FloatBuffer uvs = face.getMeshTextureCoordinates();
ShortBuffer indices = face.getMeshTriangleIndices();
// Center and region poses, mesh vertices, and normals are updated each frame.
Pose facePose = face.getCenterPose();
FloatBuffer faceVertices = face.getMeshVertices();
FloatBuffer faceNormals = face.getMeshNormals();
// Render the face using these values with OpenGL
}
}