- Sceneform SDK for Android was open sourced and archived (github.com/google-ar/sceneform-android-sdk) with version 1.16.0.
- This site (developers.google.com/sceneform) serves as the documentation archive for the previous version, Sceneform SDK for Android 1.15.0.
- Do not use version 1.17.0 of the Sceneform Maven artifacts.
- The 1.17.1 Maven artifacts can be used. Other than the version, however, the 1.17.1 artifacts are identical to the 1.15.0 artifacts.
Creating custom textures and 3D models for Augmented Faces
Stay organized with collections
Save and categorize content based on your preferences.
To implement Augmented Faces, you need customized textures and models for
overlaying on identified face meshes. These assets are created by artists
ahead of time in 3D modeling and animation software, and exported as *.fbx
files.
The SDKs ship with a canonical_face_mesh.fbx file and a
canonical_face_mesh.psd file to help artists create assets that will overlay
properly onto faces detected by ARCore. These files can be found in the following
locations for each SDK:
| SDK |
Location |
|
ARCore SDK for Android
|
assets/canonical_face_mesh.fbx
|
|
Sceneform SDK for Android
|
assets/canonical_face_mesh.fbx
|
|
ARCore SDK for iOS
|
assets/canonical_face_mesh.fbx
For iOS, note that both the `*.fbx` and `*.psd` files are included in the
public iOS GitHub repo,
but are not shipped as part of the ARCore Cocoapod.
|
|
ARCore SDK for Unity
|
/Assets/GoogleARCore/Examples/AugmentedFaces/Models/
|
Using the *.fbx file to create assets
The *.fbx file contains the face mesh topology, UV texture coordinates, and
rig that defines the supported facial regions provided for creating and
attaching assets. To ensure they're compatible with ARCore,*.fbx files should
be saved using supported settings. You can then import these files into your
project as *.sfb files.
This file contains a reference facemesh that should not be exported as part of
the final *.fbx. ARCore will generate and update a separate face mesh at
runtime.
Using the *.psd file to create assets
The .psd face mesh reference texture is used to see how a texture that an
artist creates lines up with a user's facial features at runtime. It includes
four layers:
- Mask: Shows where the eyes, nostrils, and mouth are located in the texture.
- Lines: Guidelines that show how a user's facial features line up with a
texture during runtime.
- UVs: Represents triangulation of the 468 point face texture mesh.
- Background: A neutral gray background layer to make the other three
components visually clear.
![]()
If you modify the *.psd file, you should see your changes as soon as you
export.
We recommend and support the following settings for *.fbx files exported from
modeling and animation software.
File format: FBX 2016/2017 or later
Axis conversion: Up axis is Y
Scale factor: Centimeters
Required geometry settings:
- Smoothing groups
- Smooth mesh
- Referenced asset content
Recommended geometry settings:
- Tangents and binormals
- Triangulate
- NURBS
Animation: enabled
Deformed models settings:
Customizing the canonical face mesh
If models and textures are being created using the
assets/canonical_face_mesh.fbx included in the SDK, these settings are also
required when exporting *.fbx files:
Make sure the any custom meshes are skinned to the associated bones or regions.
Use the following hierarchy:
asset
|__root
| |__NOSE_TIP
| |__FOREHEAD_RIGHT
| |__FOREHEAD_LEFT
|__facemesh <-- for reference
|__ <-- place additional custom 3D meshes here
Do not export the included facemesh model. This mesh is for reference only.
ARCore will create a separate face mesh at runtime. Use the UVs in facemesh
as a reference when creating custom face mesh textures.
Place custom meshes as children of the asset node.
Use of namespaces is supported.
The asset contains four bones consisting of root and three regions:
NOSE_TIP, FOREHEAD_RIGHT, and FOREHEAD_LEFT. Do not change the names
of these bones.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-06-26 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-06-26 UTC."],[[["\u003cp\u003eAugmented Faces requires custom 3D models and textures (\u003ccode\u003e.fbx\u003c/code\u003e files) designed by artists to overlay on detected faces.\u003c/p\u003e\n"],["\u003cp\u003eSDKs provide \u003ccode\u003ecanonical_face_mesh.fbx\u003c/code\u003e and \u003ccode\u003ecanonical_face_mesh.psd\u003c/code\u003e files to guide asset creation for proper alignment with ARCore's face detection.\u003c/p\u003e\n"],["\u003cp\u003eWhen exporting \u003ccode\u003e.fbx\u003c/code\u003e files, specific settings like FBX format, axis, scale, and geometry options are necessary for compatibility with ARCore.\u003c/p\u003e\n"],["\u003cp\u003eThe provided \u003ccode\u003efacemesh\u003c/code\u003e within the \u003ccode\u003e.fbx\u003c/code\u003e is for reference and should not be exported; ARCore generates its own face mesh at runtime.\u003c/p\u003e\n"],["\u003cp\u003eCustom meshes should be skinned to the designated bones (root, NOSE_TIP, FOREHEAD_RIGHT, FOREHEAD_LEFT) and placed under the \u003ccode\u003easset\u003c/code\u003e node in the hierarchy.\u003c/p\u003e\n"]]],["Artists create custom face assets (textures and models) in 3D software and export them as `*.fbx` files. The SDKs provide `canonical_face_mesh.fbx` and `canonical_face_mesh.psd` files to guide asset creation. `*.fbx` files contain face mesh data; the `.psd` file provides a reference texture with mask, line, and UV layers. Exported `*.fbx` files need specific settings, including proper hierarchy, skinning, and avoiding the export of the reference `facemesh`. Custom meshes should be children of the \"asset\" node.\n"],null,["# Creating custom textures and 3D models for Augmented Faces\n\nTo implement Augmented Faces, you need customized textures and models for\noverlaying on identified face meshes. These assets are created by artists\nahead of time in 3D modeling and animation software, and exported as `*.fbx`\nfiles.\n\nThe SDKs ship with a `canonical_face_mesh.fbx` file and a\n`canonical_face_mesh.psd` file to help artists create assets that will overlay\nproperly onto faces detected by ARCore. These files can be found in the following\nlocations for each SDK:\n\n| SDK | Location |\n|---------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| ARCore SDK for Android | `assets/canonical_face_mesh.fbx` |\n| Sceneform SDK for Android | `assets/canonical_face_mesh.fbx` |\n| ARCore SDK for iOS | `assets/canonical_face_mesh.fbx` For iOS, note that both the \\`\\*.fbx\\` and \\`\\*.psd\\` files are included in the [public iOS GitHub repo](//github.com/google-ar/arcore-ios-sdk/tree/master/Assets), but are not shipped as part of the ARCore Cocoapod. |\n| ARCore SDK for Unity | `/Assets/GoogleARCore/Examples/AugmentedFaces/Models/` |\n\nUsing the `*.fbx` file to create assets\n---------------------------------------\n\nThe `*.fbx` file contains the face mesh topology, UV texture coordinates, and\nrig that defines the supported facial regions provided for creating and\nattaching assets. To ensure they're compatible with ARCore,`*.fbx` files should\nbe saved using supported settings. You can then import these files into your\nproject as `*.sfb` files.\n\nThis file contains a reference `facemesh` that should not be exported as part of\nthe final `*.fbx`. ARCore will generate and update a separate face mesh at\nruntime.\n\nUsing the `*.psd` file to create assets\n---------------------------------------\n\nThe `.psd` face mesh reference texture is used to see how a texture that an\nartist creates lines up with a user's facial features at runtime. It includes\nfour layers:\n\n- **Mask:** Shows where the eyes, nostrils, and mouth are located in the texture.\n- **Lines:** Guidelines that show how a user's facial features line up with a texture during runtime.\n- **UVs:** Represents triangulation of the 468 point face texture mesh.\n- **Background:** A neutral gray background layer to make the other three components visually clear.\n\nIf you modify the `*.psd` file, you should see your changes as soon as you\nexport.\n\nFBX asset format and requirements\n---------------------------------\n\nWe recommend and support the following settings for `*.fbx` files exported from\nmodeling and animation software.\n\n- File format: FBX 2016/2017 or later\n\n- Axis conversion: Up axis is Y\n\n- Scale factor: Centimeters\n\n- Required geometry settings:\n\n - Smoothing groups\n - Smooth mesh\n - Referenced asset content\n- Recommended geometry settings:\n\n - Tangents and binormals\n - Triangulate\n - NURBS\n- Animation: enabled\n\n- Deformed models settings:\n\n - Deformed models\n - Skins\n\n### Customizing the canonical face mesh\n\nIf models and textures are being created using the\n`assets/canonical_face_mesh.fbx` included in the SDK, these settings are also\nrequired when exporting `*.fbx` files:\n\n- Make sure the any custom meshes are skinned to the associated bones or regions.\n\n- Use the following hierarchy:\n\n ```\n asset\n |__root\n | |__NOSE_TIP\n | |__FOREHEAD_RIGHT\n | |__FOREHEAD_LEFT\n |__facemesh \u003c-- for reference\n |__ \u003c-- place additional custom 3D meshes here\n ```\n- Do not export the included `facemesh` model. This mesh is for reference only.\n ARCore will create a separate face mesh at runtime. Use the UVs in `facemesh`\n as a reference when creating custom face mesh textures.\n\n- Place custom meshes as children of the `asset` node.\n\n- Use of namespaces is supported.\n\n- The asset contains four bones consisting of `root` and three regions:\n `NOSE_TIP`, `FOREHEAD_RIGHT`, and `FOREHEAD_LEFT`. Do not change the names\n of these bones."]]
