Sync different identity systems

Access control in Google Cloud Search is based on the user's Google account. When indexing content, all ACLs on items must resolve to valid Google user or group IDs (email addresses).

In many cases a repository doesn't have direct knowledge of Google accounts. Instead, users may be represented by local accounts or use federated sign-in with an identity provider and ID, other than the user's email address, to identify each account. This ID is called the external ID.

Created using the Admin console, Identity sources help bridge this gap between identity systems by:

Use identity sources when either:

  • The repository does not have knowledge of the primary email address of the user in G Suite or Google Cloud Directory.
  • The repository defines groups for access control that do not correspond to email-based groups in G Suite.

Identity sources improve indexing efficiency by decoupling indexing from identity mapping. This decoupling allows you to defer looking up the user when creating ACLs and indexing items.

Example deployment

Figure 1 shows an example deployment where both on-premise and cloud repositories are used by an enterprise. Each repository uses a different type of external ID to refer to users.

Example deployment
Figure 1. Example enterprise deployment with different identity types.

Repository 1 identifies the user using the email address asserted using SAML. Because repository 1 has knowledge of the primary email address of the user in G Suite or Cloud Directory, an identity source is not needed.

Repository 2 integrates directly with an on-premise directory and identifies the user by their sAMAccountName attribute. Because repository 2 uses a sAMAccountName attribute as an external ID, an identity source is needed.

Create an identity source

If you require an identity source, see Map user identities in Cloud Search.

You must create an identity source before creating a content connector because you will need the identity source ID to create ACLs and index data. As mentioned previously, creating an identity source also creates a custom user property in Cloud Directory. Use this property to record the external ID for each user in your repository. The property is named using the convention


The following table shows two identity sources, one to hold SAM account names (sAMAccountName) as external IDs and one to hold user IDs (uid) as external IDs.

Identity source user property external ID
id1 id1_identity sAMAccountName
id2 id2_identity uid

Create an identity source for each possible external ID that is used to refer to a user in your enterprise.

The following table shows how a user with a Google account and two external IDs (id1_identity and id2_identity) and their values appear in Cloud Directory:

user email id1_identity id2_identity
Ann example\ann 1001

You can reference the same user using the three different IDs, (Google email, sAMAccountName, and uid) when forming ACLs for indexing.

Write user ACLs

Use the getUserPrincpal() method or the getGroupPrincipal() method to create principals using a provided external ID.

The following example demonstrates how to retrieve file permissions. These permissions include the name of each user who has access to the file.
 * Sample for mapping permissions from a source repository to Cloud Search
 * ACLs. In this example, POSIX file permissions are used a the source
 * permissions.
 * @return Acl
 * @throws IOException if unable to read file permissions
static Acl mapPosixFilePermissionToCloudSearchAcl(Path pathToFile) throws IOException {
  // Id of the identity source for external user/group IDs. Shown here,
  // but may be omitted in the SDK as it is automatically applied
  // based on the `api.identitySourceId` configuration parameter.
  String identitySourceId = "abcdef12345";

  // Retrieve the file system permissions for the item being indexed.
  PosixFileAttributeView attributeView = Files.getFileAttributeView(

  if (attributeView == null) {
    // Can't read, return empty ACl
    return new Acl.Builder().build();

  PosixFileAttributes attrs = attributeView.readAttributes();
  // ...

The following code snippet shows how to create principals who are owners using the external ID (externalUserName) stored in the attributes.
// Owner, for search quality.
// Note that for principals the name is not the primary
// email address in Cloud Directory, but the local ID defined
// by the OS. Users and groups must be referred to by their
// external ID and mapped via an identity source.
List<Principal> owners = Collections.singletonList(
    Acl.getUserPrincipal(attrs.owner().getName(), identitySourceId)

Finally, the following code snippet shows how to create principals who are readers of the file.
// List of users to grant access to
List<Principal> readers = new ArrayList<>();

// Add owner, group, others to readers list if permissions
// exist. For this example, other is mapped to everyone
// in the organization.
Set<PosixFilePermission> permissions = attrs.permissions();
if (permissions.contains(PosixFilePermission.OWNER_READ)) {
  readers.add(Acl.getUserPrincipal(attrs.owner().getName(), identitySourceId));
if (permissions.contains(PosixFilePermission.GROUP_READ)) {
  String externalGroupName =;
  Principal group = Acl.getGroupPrincipal(externalGroupName, identitySourceId);
if (permissions.contains(PosixFilePermission.OTHERS_READ)) {
  Principal everyone = Acl.getCustomerPrincipal();

Once you have a list of readers and owners, you can create the ACL:
// Build the Cloud Search ACL. Note that inheritance of permissions
// from parents is omitted. See `setInheritFrom()` and `setInheritanceType()`
// methods on the builder if required by your implementation.
Acl acl = new Acl.Builder()

The underlying REST API uses the pattern

for the ID when creating principals. Referring back to the previous tables, if you create an ACL with Ann's id1_identity (SAMAccountName), the ID would resolve to:


This entire ID is called the user's intermediate ID because it provides a bridge between the external ID and the Google IDs stored with Cloud Directory.

For further information on modeling the ACLs used for a repository, see ACLs.

Map groups

Identity sources also serve as a namespace for groups used in ACLs. You can use this namespace feature to create and map groups that are used for security purposes only or are local to a repository.

Use the Cloud Identity Groups API to create a group and manage the memberships. To associate the group with an identity source, use the identity source resource name as the group namespace.

The following code snippet shows how to create a group using the Cloud Identity Groups API:
String namespace = "identitysources/" + idSource;
Group group = new Group()
    .setGroupKey(new EntityKey().setNamespace(namespace).setId(groupId))
    .setDescription("Demo group")
    .setLabels(Collections.singletonMap("system/groups/external", ""))
try {
  CloudIdentity service = Utils.buildCloudIdentityService();
  Operation createOperation = service.groups().create(group).execute();

  if (createOperation.getDone()) {
    // Note: The response contains the data for a Group object, but as
    // individual fields. To convert to a Group instance, either populate
    // the fields individually or serialize & deserialize to/from JSON.
    // Example:
    // String json = service.getJsonFactory().toString(response);
    // Group createdGroup =  service.getObjectParser()
    //     .parseAndClose(new StringReader(json), Group.class);
    System.out.printf("Group: %s\n",
  } else {
    // Handle case where operation not yet complete, poll for
    // completion. API is currently synchronous and all operations return
    // as completed.
    // ...
} catch (Exception e) {
  System.err.printf("Unable to create group: %s", e.getMessage());

Create a group ACL

To create a group ACL, use the getGroupPrincipal() method to create a group principal using a provided external ID. Then, build the ACL using the Acl.Builder class as follows:
if (permissions.contains(PosixFilePermission.GROUP_READ)) {
  String externalGroupName =;
  Principal group = Acl.getGroupPrincipal(externalGroupName, identitySourceId);

Identity connectors

While you can use external, non-Google IDs to create ACLs and index items, users can't see items in a search until their external IDs resolve to a Google ID in Cloud Directory. There are three ways to ensure that Cloud Directory knows both the Google ID and external IDs for a user:

Identity connectors are programs used to map external IDs from enterprise identities (users and groups) to internal Google identities used by Google Cloud Search. If you have to create an identity source, you must create an identity connector.

Google Cloud Directory Sync (GCDS) is an example of an identity connector. This identity connector maps user and group information from Microsoft's Active Directory to Cloud Directory along with the user attributes that may represent their identity in other systems.

Sync identities using the REST API

Use the update method to sync identities using the REST API.

Remapping identities

After remapping an item's identity to another identity, you must reindex items for the new identity to take hold. For example,

  • if you try to remove a mapping from a user or remap it to another user, the original mapping is still preserved until you reindex.
  • If you delete a mapped group that is present in an item ACL, and then create a new group with the same groupKey, the new group doesn't provide access to the item until the item is reindexed.