Unity How-to Guide: Meshing with Color

Tango devices are equipped with integrated 3D sensors that measure the distance from a device to objects in the real world. These sensors give the device the ability to understand the walls, floor, and other obstacles in the environment that your application may want to respond to. For more detailed information regarding these sensors, refer to the Depth Perception documentation.

One thing you can do with depth perception is create a 3D reconstruction of your environment. This is often called "3D scanning" or "meshing". Tango provides an API to assist you with this.

In this How-to guide, you'll create an application in Unity that allows you to create a color mesh of your environment at runtime.

If you are a more experienced Unity developer, you may also want to review the code samples in GitHub and the SDK Reference Documentation.

Preliminary steps

  1. If you haven't set up your computer to develop with Tango in Unity yet, see Getting Started With Unity.

  2. Go to the Setup for All Unity Tango Apps page and perform the basic setup operations on that page. When you have finished, return to this page and continue on with the steps below.

Change the Tango Manager settings

The following settings allow Tango Manager to build and store a mesh internally.

  1. In the Hierarchy panel, click Tango Manager.

  2. In the Tango Application (Script) component, select Enable Depth.

  3. Select Enable Video Overlay, and then change its Method option to Texture and Raw Bytes.

  4. Select Enable 3D Reconstruction (Experimental), and then change its options as follows:

    • Resolution (meters) = .05

    • Select Generate Color. This causes the internal state to remember colors and provide them to the mesh.

You can leave the other Enable 3D Reconstruction options unchanged. To learn more about them, click the down arrow below:

Change the Directional Light settings

  1. In the Hierarchy panel, click Directional Light.

  2. In the Directional Light's Light component, click the Color field, and then change it to white.

  3. The default setting for Shadow Type is Soft Shadows. Change this to No Shadows.

Add and change the Tango Delta Camera prefab

If you want to learn more about prefabs, see the Unity documentation.

  1. In the Project panel, navigate to the TangoPrefabs folder.

  2. Drag the Tango Delta Camera prefab (frustrum icon) to the Hierarchy panel.

  3. In the Tango Delta Pose Controller (Script) component, clear Character Motion.

Create a Dynamic Mesh gameObject

  1. On the GameObject menu, click Create Empty.

  2. In the Inspector panel, change the name to "Dynamic Mesh."

  3. Click Add Component, and then choose Mesh > Mesh Renderer.

  4. In the Project panel, navigate to the Assets > TangoSDK > Examples > Common > Scripts folder. It should contain a script called TangoDynamicMesh.

  5. In the Hierarchy panel, click Dynamic Mesh.

  6. Drag the TangoDynamicMesh script to a blank space in the Dynamic Mesh gameObject's Inspector panel to add it as a component.

  7. In the Project panel, navigate to the Assets > TangoSDK > Examples > ExperimentalMeshBuilderWithColor > Materials folder. It should contain a material called unlit_vertex_color.

  8. In the Hierarchy panel, click Dynamic Mesh.

  9. Drag the unlit_vertex_color material to a blank space in the Dynamic Mesh gameObject's Inspector panel.

Create a GUI Game Object

  1. On the GameObject menu, click Create Empty.

  2. In the Inspector panel, change the name to "GUI".

  3. In the Project panel, navigate to the Assets > TangoSDK > Examples > ExperimentalMeshBuilderWithColor > Scripts folder. It should contain a script called MeshBuilderWithColorGUIController.

  4. In the Hierarchy panel, click GUI.

  5. Drag the MeshBuilderWithColorGUIController script to a blank space in the GUI gameObject's Inspector panel to add it as a component.

The MeshBuilderWithColorGUIController script draws three buttons on the screen which enable your user to clear the screen, pause and resume scanning, or export the mesh to a DemoMesh.obj file.

  1. In the Project panel, navigate to the Assets > TangoSDK > Examples > Common > Scripts folder. It should contain a script called FPSCounter.

  2. In the Hierarchy panel, make sure the GUI gameObject is selected.

  3. Drag the FPSCounter script to a blank space in the GUI gameObject's Inspector panel to add it as a component.

  4. In the FPS Counter (Script) component, make the following changes:

    FPS Label Offset X = 40.

    FPS Label Offset Y = 180.

Build and run your project

  1. On the File menu, click Build and Run.

  2. In the Build Settings dialog, click Build and Run.

  3. In the Build Android dialog, enter a name for your app in the Save As field, and then click Save.

Using the meshing app

To get the best 3D mesh and texture quality, make sure your environment is well- lit. Start the app, and then slowly and steadily move the device around the room. Scan the area from different perspectives. To get a more accurate mesh, you may have to scan the same area more than once. For the best quality 3D mesh, objects should be a few meters away. Reflective or dark objects, like a mirror or a black TV, may not appear in the scan.

To remove the current mesh and create a new one, click the Clear button in the upper-right corner.

To pause scanning, click the Pause button. Click it again to resume scanning.

To export (save) the mesh, click the Export button.


Extract a mesh or mesh segments

To use the mesh, you must extract it from the Tango Application script. There are two ways to do this.

Whole mesh

TangoApplication.Tango3DRExtractWholeMesh() extracts the entire mesh. You provide arrays that can be directly assigned to a Unity Mesh's attributes: Mesh.vertices, Mesh.triangles, Mesh.colors32, Mesh.normals, and Mesh.uv,. Call this when you are done running 3D reconstruction to get the final mesh.

Mesh segments

Internally, the mesh is broken into 16 * resolution meter cube segments with indices x, y, and z. TangoApplication.Tango3DExtractMeshSegment() extracts the mesh for a mesh segment. You can subscribe to mesh segment updates by implementing the OnTango3DReconstructionGridIndicesDirty() function of the ITango3DReconstruction interface. Use these methods when you want to update your mesh in real-time.

Send feedback about...

Tango Unity SDK
Tango Unity SDK