To implement Augmented Faces, you need customized textures and models for overlaying on identified face meshes. These assets are created by artists ahead of time in 3D modeling and animation software, and exported as FBX files.
The SDKs ship with a canonical_face_mesh.fbx
file and a
canonical_face_mesh.psd
file to help artists create assets that will overlay
properly onto faces detected by ARCore. These files can be found in the
following locations for each SDK:
SDK | Location |
---|---|
ARCore SDK for Android |
assets/canonical_face_mesh.fbx
|
ARCore SDK for iOS |
assets/canonical_face_mesh.fbx For iOS, note that both the FBX and PSD files are included in the public iOS GitHub repo, but are not shipped as part of the ARCore Cocoapod. |
ARCore SDK for Unity |
/Assets/GoogleARCore/Examples/AugmentedFaces/Models/
|
Using the FBX file to create assets
The FBX file contains the face mesh topology, UV texture coordinates, and rig that defines the supported facial regions provided for creating and attaching assets. To ensure they're compatible with ARCore, FBX files should be saved using supported settings. You can then import these files into your project as SFB files.
This file contains a reference facemesh
that should not be exported as part of
the final FBX. ARCore will generate and update a separate face mesh at
runtime.
Using the PSD file to create assets
The .psd
face mesh reference texture is used to see how a texture that an
artist creates lines up with a user's facial features at runtime. It includes
four layers:
- Mask: Shows where the eyes, nostrils, and mouth are located in the texture.
- Lines: Guidelines that show how a user's facial features line up with a texture during runtime.
- UVs: Represents triangulation of the 468 point face texture mesh.
- Background: A neutral gray background layer to make the other three components visually clear.
If you modify the PSD file, you should see your changes as soon as you export.
FBX asset format and requirements
We recommend and support the following settings for FBX files exported from modeling and animation software.
File format: FBX 2016/2017 or later
Axis conversion: Up axis is Y
Scale factor: Centimeters
Required geometry settings:
- Smoothing groups
- Smooth mesh
- Referenced asset content
Recommended geometry settings:
- Tangents and binormals
- Triangulate
- NURBS
Animation: enabled
Deformed models settings:
- Deformed models
- Skins
Customizing the canonical face mesh
If models and textures are being created using the
assets/canonical_face_mesh.fbx
included in the SDK, these settings are also
required when exporting FBX files:
Make sure the any custom meshes are skinned to the associated bones or regions.
Use the following hierarchy:
asset |__root | |__NOSE_TIP | |__FOREHEAD_RIGHT | |__FOREHEAD_LEFT |__facemesh <-- for reference |__ <-- place additional custom 3D meshes here
Do not export the included
facemesh
model. This mesh is for reference only. ARCore will create a separate face mesh at runtime. Use the UVs infacemesh
as a reference when creating custom face mesh textures.Place custom meshes as children of the
asset
node.Use of namespaces is supported.
The asset contains four bones consisting of
root
andNOSE_TIP
,FOREHEAD_RIGHT
, andFOREHEAD_LEFT
regions. Do not change the names of these bones.
Comparing coordinate spaces
Since different content packages work with different coordinate spaces, it is best to check the deltas between your content creation package and the target platform when creating assets. Use the following coordinate axis asset to visualize these differences.
Importing and using the coordinate axis asset
Use the following steps to identify the correct facing axis, up axis, and scale for your setup.
Import the coordinate axis asset into your digital content creation package.
Rotate the object so that the colored arrows and axes line up:
- Red: X+
- Green: Y+
- Blue: Z+
Scale the object so that the cube represents a single unit. If the size appears to be inaccurate in your app, adjust the scale to centimeters. For example, if one unit represents one meter in the scene settings, set Scale X, Scale Y, and Scale Z of the asset by a factor of
0.01
.Export the asset using the OBJ file format.
Import the asset into your project and attach it to a region. Compare the position, size, and orientation of the asset to its expected dimensions.
Exporting assets from the canonical face mesh
When exporting assets from the canonical face mesh, the coordinate spaces may differ depending on your setup and content package. Refer to Comparing coordinate spaces to ensure that the assets are configured with the facing axis, up axis, and scale according to your content package.
A mesh is not required to be skinned to associated bones or regions. However, an unskinned mesh must be a child of the associated bone or region. Local rotation and translation offsets can then be directly applied to child meshes.
The following steps assume that all meshes have been positioned and rotated to your liking, and are ready for export.
Set the facing direction. Apply global rotations to the root joint only, instead of to the child joints or meshes. If the whole asset is facing in the opposite direction, set Up Axis of the root to
180
.Reset the region transforms to the origin. Select the regions
NOSE_TIP
,FOREHEAD_RIGHT
, andFOREHEAD_LEFT
. Set the Translate and Rotate values to0
.Scale the asset correctly. If the size appears off in your app, adjust the scale to be in centimeters. For example, if one unit represents one meter in the scene settings, select the regions
NOSE_TIP
,FOREHEAD_RIGHT
, and factor of0.01
.Export each region separately. For each region, select all corresponding meshes, then choose Export Selection.