Sync different identity systems

Access control in Google Cloud Search is based on the user's Google account. When indexing content, all ACLs on items must resolve to valid Google user or group IDs (email addresses).

In many cases a repository doesn't have direct knowledge of Google accounts. Instead, users may be represented by local accounts or use federated sign-in with an identity provider and ID, other than the user's email address, to identify each account. This ID is called the external ID.

Created using the Admin console, Identity sources help bridge this gap between identity systems by:

Defining a custom user field to store external IDs. The field is used to resolve external IDs to a Google account.
Define a namespace for security groups managed by a repository or identity provider.

Use identity sources when either:

The repository does not have knowledge of the primary email address of the user in Google Workspace or Google Cloud Directory.
The repository defines groups for access control that do not correspond to email-based groups in Google Workspace.

Identity sources improve indexing efficiency by decoupling indexing from identity mapping. This decoupling allows you to defer looking up the user when creating ACLs and indexing items.

Example deployment

Figure 1 shows an example deployment where both on-premise and cloud repositories are used by an enterprise. Each repository uses a different type of external ID to refer to users.

Repository 1 identifies the user using the email address asserted using SAML. Because repository 1 has knowledge of the primary email address of the user in Google Workspace or Cloud Directory, an identity source is not needed.

Repository 2 integrates directly with an on-premise directory and identifies the user by their sAMAccountName attribute. Because repository 2 uses a sAMAccountName attribute as an external ID, an identity source is needed.

Create an identity source

If you require an identity source, see Map user identities in Cloud Search.

You must create an identity source before creating a content connector because you will need the identity source ID to create ACLs and index data. As mentioned previously, creating an identity source also creates a custom user property in Cloud Directory. Use this property to record the external ID for each user in your repository. The property is named using the convention IDENTITY_SOURCE_ID_identity.

The following table shows two identity sources, one to hold SAM account names (sAMAccountName) as external IDs and one to hold user IDs (uid) as external IDs.

Identity source	user property	external ID
id1	id1_identity	sAMAccountName
id2	id2_identity	uid

Create an identity source for each possible external ID that is used to refer to a user in your enterprise.

The following table shows how a user with a Google account and two external IDs (id1_identity and id2_identity) and their values appear in Cloud Directory:

user	email	id1_identity	id2_identity
Ann	ann@example.com	example\ann	1001

You can reference the same user using the three different IDs, (Google email, sAMAccountName, and uid) when forming ACLs for indexing.

Write user ACLs

Use the getUserPrincpal() method or the getGroupPrincipal() method to create principals using a provided external ID.

The following example demonstrates how to retrieve file permissions. These permissions include the name of each user who has access to the file.

FilePermissionSample.java

View on GitHub

/**
 * Sample for mapping permissions from a source repository to Cloud Search
 * ACLs. In this example, POSIX file permissions are used a the source
 * permissions.
 *
 * @return Acl
 * @throws IOException if unable to read file permissions
 */
static Acl mapPosixFilePermissionToCloudSearchAcl(Path pathToFile) throws IOException {
  // Id of the identity source for external user/group IDs. Shown here,
  // but may be omitted in the SDK as it is automatically applied
  // based on the `api.identitySourceId` configuration parameter.
  String identitySourceId = "abcdef12345";

  // Retrieve the file system permissions for the item being indexed.
  PosixFileAttributeView attributeView = Files.getFileAttributeView(
      pathToFile,
      PosixFileAttributeView.class,
      LinkOption.NOFOLLOW_LINKS);

  if (attributeView == null) {
    // Can't read, return empty ACl
    return new Acl.Builder().build();
  }

  PosixFileAttributes attrs = attributeView.readAttributes();
  // ...
}

The following code snippet shows how to create principals who are owners using the external ID (externalUserName) stored in the attributes.

FilePermissionSample.java

View on GitHub

// Owner, for search quality.
// Note that for principals the name is not the primary
// email address in Cloud Directory, but the local ID defined
// by the OS. Users and groups must be referred to by their
// external ID and mapped via an identity source.
List<Principal> owners = Collections.singletonList(
    Acl.getUserPrincipal(attrs.owner().getName(), identitySourceId)
);

Finally, the following code snippet shows how to create principals who are readers of the file.

FilePermissionSample.java

View on GitHub

// List of users to grant access to
List<Principal> readers = new ArrayList<>();

// Add owner, group, others to readers list if permissions
// exist. For this example, other is mapped to everyone
// in the organization.
Set<PosixFilePermission> permissions = attrs.permissions();
if (permissions.contains(PosixFilePermission.OWNER_READ)) {
  readers.add(Acl.getUserPrincipal(attrs.owner().getName(), identitySourceId));
}
if (permissions.contains(PosixFilePermission.GROUP_READ)) {
  String externalGroupName = attrs.group().getName();
  Principal group = Acl.getGroupPrincipal(externalGroupName, identitySourceId);
  readers.add(group);
}
if (permissions.contains(PosixFilePermission.OTHERS_READ)) {
  Principal everyone = Acl.getCustomerPrincipal();
  readers.add(everyone);
}

Once you have a list of readers and owners, you can create the ACL:

FilePermissionSample.java

View on GitHub

// Build the Cloud Search ACL. Note that inheritance of permissions
// from parents is omitted. See `setInheritFrom()` and `setInheritanceType()`
// methods on the builder if required by your implementation.
Acl acl = new Acl.Builder()
    .setReaders(readers)
    .setOwners(owners)
    .build();

The underlying REST API uses the pattern identitysources/IDENTITY_SOURCE_ID/users/EXTERNAL_ID for the ID when creating principals. Referring back to the previous tables, if you create an ACL with Ann's id1_identity (SAMAccountName), the ID would resolve to:

identitysources/id1_identity/users/example/ann

This entire ID is called the user's intermediate ID because it provides a bridge between the external ID and the Google IDs stored with Cloud Directory.

For further information on modeling the ACLs used for a repository, see ACLs.

Map groups

Identity sources also serve as a namespace for groups used in ACLs. You can use this namespace feature to create and map groups that are used for security purposes only or are local to a repository.

Use the Cloud Identity Groups API to create a group and manage the memberships. To associate the group with an identity source, use the identity source resource name as the group namespace.

The following code snippet shows how to create a group using the Cloud Identity Groups API:

CreateGroupCommand.java

View on GitHub

String namespace = "identitysources/" + idSource;
Group group = new Group()
    .setGroupKey(new EntityKey().setNamespace(namespace).setId(groupId))
    .setDescription("Demo group")
    .setDisplayName(groupName)
    .setLabels(Collections.singletonMap("system/groups/external", ""))
    .setParent(namespace);
try {
  CloudIdentity service = Utils.buildCloudIdentityService();
  Operation createOperation = service.groups().create(group).execute();

  if (createOperation.getDone()) {
    // Note: The response contains the data for a Group object, but as
    // individual fields. To convert to a Group instance, either populate
    // the fields individually or serialize & deserialize to/from JSON.
    //
    // Example:
    // String json = service.getJsonFactory().toString(response);
    // Group createdGroup =  service.getObjectParser()
    //     .parseAndClose(new StringReader(json), Group.class);
    System.out.printf("Group: %s\n",
        createOperation.getResponse().toString());
  } else {
    // Handle case where operation not yet complete, poll for
    // completion. API is currently synchronous and all operations return
    // as completed.
    // ...
  }
} catch (Exception e) {
  System.err.printf("Unable to create group: %s", e.getMessage());
  e.printStackTrace(System.err);
}

Create a group ACL

To create a group ACL, use the getGroupPrincipal() method to create a group principal using a provided external ID. Then, build the ACL using the Acl.Builder class as follows:

FilePermissionSample.java

View on GitHub

if (permissions.contains(PosixFilePermission.GROUP_READ)) {
  String externalGroupName = attrs.group().getName();
  Principal group = Acl.getGroupPrincipal(externalGroupName, identitySourceId);
  readers.add(group);
}

Identity connectors

While you can use external, non-Google IDs to create ACLs and index items, users can't see items in a search until their external IDs resolve to a Google ID in Cloud Directory. There are three ways to ensure that Cloud Directory knows both the Google ID and external IDs for a user:

Manually update each individual user profile through the Admin console This process is only recommended for testing and prototyping using a few user profiles.
Map external IDs to Google IDs using the Directory API. This process is recommended for those who can't use the Identity Connector SDK.
Create an identity connector using the Identity Connector SDK. This SDK simplifies the use of the Directory API to map IDs.

Identity connectors are programs used to map external IDs from enterprise identities (users and groups) to internal Google identities used by Google Cloud Search. If you have to create an identity source, you must create an identity connector.

Google Cloud Directory Sync (GCDS) is an example of an identity connector. This identity connector maps user and group information from Microsoft's Active Directory to Cloud Directory along with the user attributes that may represent their identity in other systems.

Sync identities using the REST API

Use the update method to sync identities using the REST API.

Remapping identities

After remapping an item's identity to another identity, you must reindex items for the new identity to take hold. For example,

if you try to remove a mapping from a user or remap it to another user, the original mapping is still preserved until you reindex.
If you delete a mapped group that is present in an item ACL, and then create a new group with the same groupKey, the new group doesn't provide access to the item until the item is reindexed.