Google is committed to advancing racial equity for Black communities. See how.

Cloud Anchors Developer Guide for iOS

Learn how to use the Cloud Anchors API in your own apps. If you are new to Cloud Anchors:

Enable Cloud Anchors in your app

If you create a GARSession in ARCore, Cloud Anchors are enabled by default. You can also enable the ARCore Cloud Anchors API for your Google Cloud Platform project.

Host and resolve anchors

You can host and resolve cloud anchors with the Cloud Anchors API. The API also includes delegate methods that provide callbacks on completed requests.

Host an anchor

Hosting an ARAnchor puts the anchor in a common coordinate system for any given physical space.

A host request sends visual data to a Google server, which maps the ARAnchor’s position in a coordinate system that represents the current physical space. This request returns a GARAnchor without an ID. A successful host request assigns the returned GARAnchor a unique ID in a later GARFrame.

- (void)addAnchorWithTransform:(matrix_float4x4)transform {
  self.arAnchor = [[ARAnchor alloc] initWithTransform:transform];
  [self.sceneView.session addAnchor:self.arAnchor];
  self.garAnchor = [self.gSession hostCloudAnchor:self.arAnchor error:nil];
  [self enterState:HelloARStateHosting];
}

Resolve an anchor

Resolving an ARAnchor allows Android and iOS devices in a given physical space to add previously hosted anchors to new scenes.

A resolve request sends a Google server a cloud anchor ID along with visual data from the current frame. This request returns a GARAnchor without a valid transform or cloud anchor ID. The server will attempt to match this visual data with the imagery of where currently hosted Cloud Anchors are mapped. A successful host request assigns the GARanchor a valid transform and a unique ID in a later GARFrame.

- (void)resolveAnchorWithIdentifier:(NSString *)identifier {
  self.garAnchor = [self.gSession resolveCloudAnchorWithIdentifier:identifier error:nil];
}

// Pass the ARFRame to the ARCore session every time there is a frame update.
// This returns a GARFrame that contains a list of updated anchors. If your
// anchor's pose or tracking state changed, your anchor will be on the list.
- (void)cloudAnchorManager:(CloudAnchorManager *)manager didUpdateFrame:(GARFrame *)garFrame {
  for (GARAnchor *garAnchor in garFrame.updatedAnchors) {
    if ([garAnchor isEqual:self.garAnchor] && self.resolvedAnchorNode) {
      self.resolvedAnchorNode.simdTransform = garAnchor.transform;
      self.resolvedAnchorNode.hidden = !garAnchor.hasValidTransform;
    }
  }
}

Delegate methods for host and resolve requests

Host and resolve requests have GARSessionDelegate methods that provide callbacks upon request success and failure.

Host

  • session:didHostAnchor:
  • session:didFailToHostAnchor:

Resolve

  • session:didResolveAnchor:
  • session:didFailToResolveAnchor:

-(void)session:(ARSession *)arSession didUpdateFrame:(ARFrame *)arFrame {
  [...]

-(void)session:(GARSession *)garSession didHostAnchor:(GARAnchor *)garAnchor {
  // successful host
}

-(void)session:(GARSession *)garSession didFailToHostAnchor:(GARAnchor *)garAnchor {
  // failed host
}

-(void)session:(GARSession *)garSession didResolveAnchor:(GARAnchor *)garAnchor {
  // successful resolve
}

-(void)session:(GARSession *)garSession didFailToResolveAnchor:(GARAnchor *)garAnchor {
  // failed resolve
}

Optional GARSession polling pattern

If you are using Metal or need a polling option, and your app runs at a minimum of 30 fps, use the following pattern to pass ARFrames to the GARSession:

-(void)myOwnPersonalUpdateMethod {
        ARFrame *arFrame = arSession.currentFrame;
        NSError *error = nil;
        GARFrame *garFrame = [garSession update:arFrame error:&error];
        // your update code here
}

Host a Cloud Anchor with persistence

Prior to ARCore v1.20, Cloud Anchors could only be resolved for up to 24 hours after they were first hosted. With persistent Cloud Anchors, you can now use hostCloudAnchor:error: to create a Cloud Anchor with a time to live (TTL) between one and 365 days. You can also extend the lifetime of the anchor after it is already hosted using the Cloud Anchor Management API.

/**
* This creates a new Cloud Anchor with a given lifetime in days, using the transform
 * of the provided anchor.
 *
 * The cloud state of the returned anchor will be set to GARCloudAnchorStateTaskInProgress and the
 * initial transform will be set to the transform of the provided anchor. However, the returned
 * anchor and the original anchor are independent of one another, and their two transforms
 * may diverge over time.
 *
 * Hosting requires a working Internet connection and an active session where the tracking state
 * is ARTrackingStateNormal. If it is unable to establish a connection to the ARCore Cloud Anchor
 * service, ARCore will continue to silently retry in the background.
 *
 * @param anchor The ARAnchor with the desired transform to be used to create a hosted Cloud
 *     Anchor.
 * @param TTLDays The anchor’s lifetime in days. Must be a positive number. The maximum
 * allowed value is 1 if you are using an API key to authenticate the Cloud Anchor.
 * Otherwise, the maximum allowed value is 365.
 * @param error Out parameter for an NSError. Possible errors include:
 *     GARSessionErrorCodeInvalidArgument - Invalid (nil) anchor or invalid TTL.
 *     GARSessionErrorCodeNotTracking - Bad current ARTrackingState.
 *     GARSessionErrorCodeResourceExhausted - ARCore tried to create too many Cloud Anchors.
 * @return The new GARAnchor, or nil if there is an error.
 */

- (GARAnchor *_Nullable)hostCloudAnchor:(ARAnchor *)anchor
                                TTLDays:(NSInteger)TTLDays
                                  error:(NSError **)error;

Authentication

Your app needs a form of authentication to use Cloud Anchors. You can authenticate with either the Authentication Token or the API Key method. Cloud Anchors with a TTL greater than one day must use a token for authentication.

Token (signed JWT) authentication

The authentication token option can host a Cloud Anchor for up to 365 days. Cloud Anchors with a TTL greater than one day must use this method. In order to generate tokens for iOS, you must have an endpoint on your server that satisfies the following requirements:

  • Your own authentication mechanism must protect the endpoint.

  • The endpoint must generate a new token every time, such that:

    • Each user gets a unique token.
    • Tokens don’t immediately expire.

Currently, the only supported token type is a signed JWT (that is, a JSON Web token signed by a Google Service account). See the official JWT website for an introduction to JWTs.

Create a service account and signing key

Follow these steps to create a Google Service account and signing key:

  1. In the navigation menu of the Google Cloud Platform console, go to APIs & Services > Credentials.

  2. Select the desired project, then click Create Credentials > Service account.

  3. Under Service account details, type a name for the new account, then click Create.

  4. On the Service account permissions page, go to the Select a role dropdown. Select Service Accounts > Service Account Token Creator, then click Continue.

  5. On the Grant users access to this service account page, click Done. This takes you back to APIs & Services > Credentials.

  6. On the Credentials page, scroll down to the Service Accounts section and click the name of the account you just created.

  7. On the Service account details page, scroll down to the Keys section and select Add Key > Create new key.

  8. Select JSON as the key type and click Create. This downloads a JSON file containing the private key to your machine. Store the downloaded JSON key file in a secure location.

Create tokens on your server

To create new tokens (JWTs) on your server, use the standard JWT libraries and the JSON file that you securely downloaded from your new service account.

Create tokens on your development machine

To generate JWTs on your development machine, use the following oauth2l command:

oauth2l fetch --jwt --json $KEYFILE $AUDIENCE --cache ""

Specifying an empty cache location using the --cache flag is necessary to ensure that a different token is produced each time. Be sure to trim the resulting string because extra spaces or newline characters will cause ARCore to reject the token.

Sign the token

You must use the RS256 algorithm and the following claims to sign the JWT:

  • iss — The service account email address.
  • sub — The service account email address.
  • iat — The Unix time when the token was generated, in seconds.
  • expiat + 3600 (1 hour). The Unix time when the token expires, in seconds.
  • aud — The audience. The correct ‘audience’ for the Cloud Anchors API is https://arcorecloudanchor.googoleapis.com/.

Non-standard claims are not required in the JWT payload, though you may find the uid claim useful for identifying the corresponding user.

If you use a different approach to generate your JWTs, such as using a Google API in a Google-managed environment, make sure to sign your JWTs with the claims in this section. Above all, make sure that the audience is correct.

Pass a token into the ARCore session

First, construct a session using sessionWithError::

NSError *error = nil;
GARSession *session = [GARSession sessionWithError:&error];

When you obtain a token, pass it into the session using setAuthToken:.

/**
 * Provide an auth token to use when authenticating with the ARCore Cloud Anchor service. If
 * you used an API key to create the session, ARCore will ignore the token and log an error.
 * Otherwise, it will use the most recent valid auth token that you passed in. Call this
 * method each time you refresh your token.
 *
 * @param authToken The token to use when authenticating with the ARCore Cloud Anchor service. This
 *                  must be a nonempty ASCII string with no spaces or control characters. ARCore
 *                  will use this until you pass in another token. Currently, JWTs are the only
 *                  supported token types.
 */
- (void)setAuthToken:(NSString *)authToken;

Note the following when you pass a token into the session:

  • If you do not pass in a valid token before attempting to host or resolve an anchor, you will get authentication errors.

  • ARCore ignores tokens that contain spaces or special characters. ARCore also ignores all tokens if you create your session with a valid API key. If you previously used an API key and no longer need it, we recommend deleting it in the Google Developers Console and removing it from your app after migrating users to the newest version.

  • Tokens typically expire after one hour. If there is a possibility that your token may expire while in use, obtain a new token and pass it to the API.

API key authentication

The API key authentication option can host a Cloud Anchor for up to one day.

Follow these steps to obtain and add an API key to your project:

  1. See the Google Cloud Platform Console Help Center to obtain an API key.

  2. In Xcode, pass the new API key to sessionWithAPIKey:bundleIdentifier:error: to add it to your project:

    NSError *error = nil;
    GARSession *session = [GARSession sessionWithAPIKey:@"your-api-key" bundleIdentifier:nil error:&error];
    

Mapping quality

The mapping quality API estimates the quality of the visual feature points seen by ARCore in the preceding few seconds and visible from the provided camera transform. Cloud Anchors hosted using higher quality features will generally result in easier and more accurately resolved Cloud Anchor transforms. If feature map quality cannot be estimated for a given transform, ARCore will log a warning message and return GARFeatureMapQualityInsufficient. This state indicates that ARCore will likely have more difficulty resolving the Cloud Anchor. Encourage the user to move the device, so that the desired position of the Cloud Anchor to be hosted is viewed from different angles.

/**
 * @param transform The camera transform to use to estimate the mapping quality.
 * @param error Out parameter for an `NSError`. Possible errors:
 *        GARSessionErrorCodeNotTracking - Bad current ARTrackingState.
 * @return The estimated quality of the visual feature points seen
 *        by ARCore in the preceding few seconds and visible from the provided camera
 *        transform.
 */

- (GARFeatureMapQuality)estimateFeatureMapQualityForHosting:(simd_float4x4)transform
                                                      error:(NSError **)error;

API quotas

The ARCore Cloud Anchors API has the following quotas for request bandwidth:

Quota type Maximum Duration Applies to
Number of anchors Unlimited N/A Project
Anchor host requests 30 minute IP address and project
Anchor resolve requests 300 minute IP address and project

Known issues and workarounds

There are a few known issues when working with the iOS Cloud Anchors API.

Default scheme settings cause intermittent app crash

GPU Frame Capture and Metal API Validation scheme settings are enabled by default, which can sometimes cause the app to crash within the SDK.

Diagnose an app crash

Whenever you suspect that a crash has occurred, take a look at your stack trace. If you see MTLDebugComputeCommandEncoder in the stack trace, it is likely due to the default scheme settings.

Workaround

  1. Go to Product > Scheme > Edit Scheme….

  2. Open the Run tab.

  3. Click Options to view your current settings.

  4. Make sure that both GPU Frame Capture and Metal API Validation are disabled.

  5. Build and run your app.

See the Cocoapods CHANGELOG for additional known issues.

Limitations

The ARCore SDK for iOS does not support the ARKit setWorldOrigin(relativeTransform:) method call.

Performance considerations

Memory usage increases when you enable the Cloud Anchors API. Expect the device’s battery usage to rise due to higher network usage and CPU utilization.

Next steps