# Augmented Faces developer guide for Sceneform

[Video](https://www.youtube.com/watch?v=-4EvaCQpVEQ)

Learn how to use the Augmented Faces feature in your own apps.

## Build and run the sample app

To build and run the **AugmentedFaces Java** app:

1. Open Android Studio version 3.1 or greater. It is recommended to use a
   physical device (and not the Android Emulator) to work with Augmented Faces.
   The device should be connected to the development machine via USB. See the
   [Android quickstart](https://developers.google.com/sceneform/develop/android-quickstart) for detailed steps.

2. Import the [AugmentedFaces Java sample](https://github.com/google-ar/sceneform-android-sdk/tree/v1.15.0/samples/augmentedfaces)
   into your project.

3. In Android Studio, click **Run** ![](https://developers.google.com/static/ar/images/toolbar-run.png). Then,
   choose your device as the deployment target and click **OK** to launch the
   sample app on your device.

4. Click **Approve** to give the camera access to the sample app.

   The app should open the front camera and immediately track your face in the
   camera feed. It should place images of fox ears over both sides of your
   forehead, and place a fox nose over your own nose.

## Using Augmented Faces in Sceneform

1. [Import assets into Sceneform](https://developers.google.com/sceneform/develop/augmented-faces/developer-guide#import-assets)

2. [Configure the ARCore session](https://developers.google.com/sceneform/develop/augmented-faces/developer-guide#configure-session)

3. [Get access to the detected face](https://developers.google.com/sceneform/develop/augmented-faces/developer-guide#get-access)

4. [Render the effect on the detected face](https://developers.google.com/sceneform/develop/augmented-faces/developer-guide#render-effect)

### Import assets into Sceneform

Make sure that assets you use for Augmented Faces are scaled and positioned
correctly. For tips and practices, refer to [Creating Assets for Augmented Faces](https://developers.google.com/sceneform/develop/augmented-faces/creating-assets).

To apply assets such as textures and 3D models to an augmented face mesh in Sceneform,
first import the assets.

At runtime, use [`ModelRenderable.Builder`](https://developers.google.com/sceneform/reference/com/google/ar/sceneform/rendering/ModelRenderable.Builder)
to load the `*.sfb` models, and use the [`Texture.Builder`](https://developers.google.com/sceneform/reference/com/google/ar/sceneform/rendering/Texture.Builder)
to load a texture for the face.

    // To ensure that the asset doesn't cast or receive shadows in the scene,
    // ensure that setShadowCaster and setShadowReceiver are both set to false.
    ModelRenderable.builder()
        .setSource(this, R.raw.fox_face)
        .build()
        .thenAccept(
            modelRenderable -> {
              faceRegionsRenderable = modelRenderable;
              modelRenderable.setShadowCaster(false);
              modelRenderable.setShadowReceiver(false);
            });

    // Load the face mesh texture.
    Texture.builder()
        .setSource(this, R.drawable.fox_face_mesh_texture)
        .build()
        .thenAccept(texture -> faceMeshTexture = texture);

### Face mesh orientation

Note the orientation of the face mesh for Sceneform:

### Configure the ARCore session

Augmented Faces requires the ARCore session to be configured to use the
front-facing (selfie) camera and enable face mesh support. To do this in
Sceneform, extend the [ARfragment](https://developers.google.com/sceneform/reference/com/google/ar/sceneform/ux/ArFragment)
class, and override the configuration:

    @Override
    protected Set<Session.Feature> getSessionFeatures() {
      return EnumSet.of(Session.Feature.FRONT_CAMERA);
    }

    @Override
    protected Config getSessionConfiguration(Session session) {
      Config config = new Config(session);
      config.setAugmentedFaceMode(AugmentedFaceMode.MESH3D);
      return config;
    }

Refer to this subclassed `ArFragment` class in your activity layout.

### Get access to the detected face

The `AugmentedFace` class extends the `Trackable` class. In your app's activity,
use `AugmentedFace` to get access to the detected face by calling it from the [`addOnUpdateListener()`](https://developers.google.com/sceneform/reference/com/google/ar/sceneform/Scene#public-void-addonupdatelistener-scene.onupdatelistener-onupdatelistener)
method.

    // Get list of detected faces.
    Collection<AugmentedFace> faceList = session.getAllTrackables(AugmentedFace.class);

### Render the effect for the face

Rendering the effect involves these steps:

    for (AugmentedFace face : faceList) {
      // Create a face node and add it to the scene.
      AugmentedFaceNode faceNode = new AugmentedFaceNode(face);
      faceNode.setParent(scene);

      // Overlay the 3D assets on the face.
      faceNode.setFaceRegionsRenderable(faceRegionsRenderable);

      // Overlay a texture on the face.
      faceNode.setFaceMeshTexture(faceMeshTexture);

      ...
     }