Google is committed to advancing racial equity for Black communities. See how.

Record and play back an AR session on AR Foundation targeting Android

The Recording & Playback API enables you to record video and AR data once within a given environment and use that content to replace a live camera session.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

Compatibility with other ARCore APIs

Due to the way session data is processed, ARCore APIs may produce different results during playback than observed during recording. They may also produce different results during subsequent playback sessions. For example, the number of detected trackables, the precise timing of their detection, and their poses over time may be different during playback.

Compatibility with Cloud Anchors

You can host and resolve Cloud Anchors while recording or playing back a session.

Recording

Start, stop, and check the status of an ARCore session recording.

Record an ARCore session

To record an ARCore session, configure the session and provide an MP4 path for the recording. Call ARRecordingManager.StartRecording() before resuming the session. Recording automatically starts when the session resumes. To automatically stop recording when the session is paused, call ARRecordingConfig.AutoStopOnPause. To record a partial session, call ARRecordingManager.StartRecording() while the session is running.

ARCoreRecordingConfig recordingConfig = ScriptableObject.CreateInstance<ARCoreRecordingConfig>();
recordingConfig.Mp4DatasetFilepath = "path/to/file.mp4";

recordingManager.StartRecording(recordingConfig);

Stop a session recording

To stop recording without pausing the currently running AR session, call ARRecordingManager.StopRecording().

recordingManager.StopRecording();

Check recording status

ARRecordingManager.RecordingStatus can be used at any time to determine the current recording status.

Debug.Log("Current Recording Status: " + recordingManager.RecordingStatus);

Playback

Play back previously recorded AR sessions. Sessions play back in real time, and session playback or speed cannot be adjusted.

Playback a previously recorded session

To play back a previously recorded session, call ARPlaybackManager.SetPlaybackDataset() and provide a file path for the dataset you wish to play back. You must pause the session to use this method. Resume the session for the change to take effect.

Once playback has started due to resuming the session, pausing the session by disabling theARSession will suspend processing of all camera image frames and any other recorded sensor data in the dataset. Camera image frames and sensor frame data that is discarded in this way will not be reprocessed when the session is again resumed by resuming the session. AR tracking for the session will generally suffer due to the gap in processed data.

// Disable the ARSession to pause the current AR session.
session.enabled = false;

// In the next frame, provide a file path for the dataset you wish to play back.
playbackManager.SetPlaybackDataset("path/to/file.mp4");

// In the frame after that, re-enable the ARSession to resume the session from
// the beginning of the dataset.
session.enabled = true;

Known issue and workaround

There is a known issue where calls to ARPlaybackManager.SetPlaybackDataset() return ErrorPlaybackFailed. This happens because it can take several frames for a session to be paused. If ARPlaybackManager.SetPlaybackDataset() is called before the session has been paused, it will not be able to access the session, so it will return an error.

The following code can be used as a workaround solution.

// Workaround for known issue where `playbackManager.SetPlaybackDataset()`
// returns `ErrorPlaybackFailed` because it can take several frames for a
// session to be paused.

// Reference to the ARSession component in the scene.
ARSession session;

void PlaybackDataset()
{
    setPlaybackDataset = true;

    // Pause the current AR session.
    session.enabled = false;

    // Set a timeout for retrying playback retrieval.
    timeout = 10f;
}

// Next frame
void Update()
{
    ...

    if (setPlaybackDataset)
    {
        PlaybackResult result = playbackManager.SetPlaybackDataset("path/to/file.mp4");
        if (result == PlaybackResult.ErrorPlaybackFailed || result == PlaybackResult.SessionNotReady)
        {
            // Try to set the dataset again in the next frame.
            timeout -= Time.deltaTime;
        }
        else
        {
            // Do not set the timeout if the result is something other than ErrorPlaybackFailed.
            timeout = -1f;
        }

        if (timeout < 0.0f)
        {
            setPlaybackDataset = false;
            // If playback is successful, proceed as usual.
            // If playback is not successful, handle the error appropriately.
        }
    }

    ...
}

Stop a playback

To stop a playback, call ARPlaybackManager.SetPlaybackDataset() and set the dataset filepath to null.

// Disable the ARSession to pause the current AR session.
session.enabled = false;

// In the next frame, unset the playback dataset file path.
playbackManager.SetPlaybackDataset(null);

// In the frame after that, re-enable the ARSession to resume the session using
// the device camera and other sensors.
session.enabled = true;

Restart playback from the beginning

To restart a playback from the beginning of the dataset, call ARPlaybackManager.SetPlaybackDataset() and specify the same MP4 recording before resuming the session.

// Disable the ARSession to pause the current AR session.
session.enabled = false;

// In the next frame, specify the same dataset file path.
playbackManager.SetPlaybackDataset(filepath); // Same path that was previously set.

// In the frame after that, re-enable the ARSession to resume the session from
// the beginning of the dataset.
session.enabled = true;

Play back a different session

To play back a different dataset, pause the session and specify the new dataset before resuming the session.

// Disable the ARSession to pause the current AR session.
session.enabled = false;

// In the next frame, specify a new dataset file path.
playbackManager.SetPlaybackDataset(newFilepath); // Different path than was previously set.

// In the frame after that, re-enable the ARSession to resume the session from
// the beginning of the new dataset.
session.enabled = true;

Check playback status

ARPlaybackManager.PlaybackStatus can be used at any time to determine the current playback status.

Debug.Log("Current Playback Status: " + playbackManager.PlaybackStatus);

What’s next