AugmentedFace

public class AugmentedFace

Describes a face detected by ARCore and provides methods to access additional center and face region poses as well as face mesh related data.

Augmented Faces supports front-facing (selfie) camera only, and does not support attaching anchors nor raycast hit testing. Calling Trackable.createAnchor(Pose) will result in an IllegalStateException.

To use Augmented Faces, enable the feature in the session. This can be done at session creation time, or at any time during session runtime:

Session session = new Session(context, EnumSet.of(Session.Feature.FRONT_CAMERA));
 Config config = ...
 config.setAugmentedFaceMode(AugmentedFaceMode.MESH3D);
 session.configure(config);
 
When Augmented Face mode is enabled, ARCore updates the list of detected faces for each frame. Use Session.getAllTrackables(Class) and Trackable.getTrackingState() to get a list of faces that have valid meshes that can be rendered.
for (AugmentedFace face : session.getAllTrackables(AugmentedFace.class)) {
   if (face.getTrackingState() == TrackingState.TRACKING) {
     // Render face mesh ...
   }
 }
 
Faces provide static mesh data that does not change during the session, as well as pose and mesh data that is updated each frame:
// UVs and indices can be cached as they do not change during the session.
 FloatBuffer uvs = face.getMeshTextureCoordinates();
 ShortBuffer indices = face.getMeshTriangleIndices();
 // Center and region poses, mesh vertices, and normals are updated each frame.
 Pose facePose = face.getCenterPose();
 FloatBuffer faceVertices = face.getMeshVertices();
 FloatBuffer faceNormals = face.getMeshNormals();
 

Nested Classes

enum AugmentedFace.RegionType Defines face regions to query the pose for. 

Public Methods

Anchor
createAnchor(Pose pose)
Creates an anchor that is attached to this trackable, using the given initial pose in the world coordinate space.
boolean
equals(Object obj)
Indicates whether some other object is a Trackable referencing the same logical trackable as this one.
Collection<Anchor>
getAnchors()
Gets the Anchors attached to this trackable.
Pose
getCenterPose()
Returns the pose of the center of the face, defined to have the origin located behind the nose and between the two cheek bones.
FloatBuffer
getMeshNormals()
Returns a float buffer of 3D normals in (x, y, z) packing.
FloatBuffer
getMeshTextureCoordinates()
Returns a float buffer of UV texture coordinates in (u, v) packing.
ShortBuffer
getMeshTriangleIndices()
Returns a char buffer of triangles' indices in consecutive triplets.
FloatBuffer
getMeshVertices()
Returns a float buffer of 3D vertices in (x, y, z) packing.
Pose
getRegionPose(AugmentedFace.RegionType regionType)
Returns the pose of a region in world coordinates when the face trackable state is TRACKING.
TrackingState
getTrackingState()
Gets this trackable's TrackingState.
int
hashCode()
Returns a hash code value for the object.

Inherited Methods

Public Methods

createAnchor

public Anchor createAnchor(
  Pose pose
)

Creates an anchor that is attached to this trackable, using the given initial pose in the world coordinate space. The type of trackable will determine the semantics of attachment and how the anchor's pose will be updated to maintain this relationship. Note that the relative offset between the pose of multiple anchors attached to a trackable may adjust slightly over time as ARCore updates its model of the world.

Details
Parameters
pose

equals

public boolean equals(
  Object obj
)

Indicates whether some other object is a Trackable referencing the same logical trackable as this one.

Details
Parameters
obj the reference object with which to compare.
Returns true if this object is the same as the obj argument; false otherwise.
See Also

getAnchors

public Collection<Anchor> getAnchors()

Gets the Anchors attached to this trackable.

getCenterPose

public Pose getCenterPose()

Returns the pose of the center of the face, defined to have the origin located behind the nose and between the two cheek bones. Z+ is forward out of the nose, Y+ is upwards, and X+ is towards the left. The units are in meters. When the face trackable state is TRACKING, this pose is synced with the latest frame. When face trackable state is PAUSED, an identity pose will be returned.

Use getRegionPose(AugmentedFace.RegionType) to retrieve poses of specific regions of the face.

getMeshNormals

public FloatBuffer getMeshNormals()

Returns a float buffer of 3D normals in (x, y, z) packing.

Each (x, y, z) is a unit vector of the normal to the surface at each vertex. There is exactly one normal vector for each vertex. These normals are relative to the center pose of the face. When the face trackable state is TRACKING, this pose is synced with the latest frame. While face trackable state is PAUSED the returned buffer size will be 0.

Details
Returns a FloatBuffer of normals when TRACKING, or an empty FloatBuffer when PAUSED.

getMeshTextureCoordinates

public FloatBuffer getMeshTextureCoordinates()

Returns a float buffer of UV texture coordinates in (u, v) packing.

There is a pair of texture coordinates for each vertex. These values do not change over a session, however while face trackable state is PAUSED the returned buffer size will be 0.

Details
Returns a FloatBuffer of texture coordinates when TRACKING, or an empty FloatBuffer when PAUSED.

getMeshTriangleIndices

public ShortBuffer getMeshTriangleIndices()

Returns a char buffer of triangles' indices in consecutive triplets.

Every three consecutive values are indices that represent a triangle. The vertex position and texture coordinates are mapped by the indices. The front face of each triangle is defined by the face where the vertices are in counter clockwise winding order. These values do not change, however while face trackable state is PAUSED the returned buffer size will be 0.

Details
Returns a ShortBuffer of triangle indices when TRACKING, or an empty ShortBuffer when PAUSED.

getMeshVertices

public FloatBuffer getMeshVertices()

Returns a float buffer of 3D vertices in (x, y, z) packing.

These vertices are relative to the center pose of the face with units in meters. When the face trackable state is TRACKING, this data is synced with the latest frame. While face trackable state is PAUSED the returned buffer size will be 0.

Details
Returns a FloatBuffer of vertices when TRACKING, or an empty FloatBuffer when PAUSED.

getRegionPose

public Pose getRegionPose(
  AugmentedFace.RegionType regionType
)

Returns the pose of a region in world coordinates when the face trackable state is TRACKING. When face trackable state is PAUSED, an identity pose will be returned.

Details
Parameters
regionType The face region for which to get the pose.
Returns the Pose of the selected region when TRACKING, or an identity pose when PAUSED.

getTrackingState

public TrackingState getTrackingState()

Gets this trackable's TrackingState.

hashCode

public int hashCode()

Returns a hash code value for the object. This method is supported for the benefit of hash tables such as those provided by HashMap.

Details
Returns a hash code value for this object.
See Also