Create an experiment integration with Google Analytics

Page Summary

• This guide is for third-party A/B experiment solution providers wanting to integrate their tools with Google Analytics.

• Integrating requires using Google Analytics events to add users to variants, the Google Analytics Admin API to create audiences for each variant, and OAuth 2.0 for user consent.

• You need to add a formatted exp_variant_string parameter to the experience_impression event to identify variants and associate users with them.

• Audiences in Google Analytics are used to group users based on the exp_variant_string parameter, and the Google Analytics Admin API is used to create and archive these audiences.

• Audience targeting allows customers to focus experiments on specific user groups by comparing User IDs or device identifiers from Google Analytics with those in your system using the Audience List API.

This guide explains how to integrate a third-party A/B experiment tool with Google Analytics. The integration lets your users run experiment variants in your tool and use Google Analytics to interpret the results.

Who this is for

The guide is primarily intended for third-party A/B experiment solution providers.

• If you want to learn about the integration between Google Analytics and third-party A/B experiment solutions, see Integrating with a third-party experiment tool.
• If you are developing a custom or in-house experiment framework instead of integrating a third-party tool, see In-house and custom experiments.

Overview

The integration between your third-party experiment tool and Google Analytics requires you to use:

• Google Analytics events to add users to a variant
• Google Analytics Admin API to create an audience for the users who have been added to each variant
• OAuth 2.0 to get user consent to creating audiences on behalf of the user. Learn more about OAuth 2.0

Additionally, the user who starts the experiment in your tool must be an Editor (or above) in the Google Analytics property.

Add the exp_variant_string parameter to an event

You need to create an identifier for each variant and then add the identifier to the exp_variant_string parameter in the following experience_impression event. You'll use the parameter to create an audience to associate a user with a variant.

gtag('event', 'experience_impression', {
  // Replace the value with the Experiment-variant ID
  exp_variant_string: "ABC-F2948574-3495F49"
});

Send the experience_impression event when a user is added to a variant, such as when an experiment page loads.

Format the parameter

To avoid duplication across experiments and other partners, we recommend using the format TOOL_ID-EXPERIENCE_ID-VARIANT_ID for the exp_variant_string parameter value, where:

• TOOL_ID is the ID for your third-party tool
• EXPERIENCE_ID is the ID of the experience
• VARIANT_ID is the ID of the variant

You can use any number of characters for your tool, experiences, and variants.

Define an audience using the exp_variant_string parameter

Audiences allow you to group users who have been added to a variant, based on the exp_variant_string parameter. By using this parameter as the membership condition for an audience, you can add users to the appropriate audience based on the variant they have been added to.

To create audiences for new experiences and archive audiences for ended experiences, use the properties.audiences resource in the Google Analytics Admin API.

Create audiences

The following request body creates an audience for users who have been added to a variant with the ID ABC-F2948574-3495F49 using the properties.audiences.create method. Note that you can adjust the membership duration based on the length of the experience (max: 540).

We recommend that you set the atAnyPointInTime parameter to true to reduce data discrepancies across experiments. When you set the parameter to true, a user who has been assigned to a passed experiment can be assigned to a new experiment. When you set the parameter to false or if it's unset, the user cannot be added to a new experiment if they've already been assigned to a passed experiment.

{
  "displayName": "Audience ABC - experiment F2948574 variant 3495F49",
  "membershipDurationDays": 30,
  "filterClauses": [
    {
      "clauseType": "INCLUDE",
      "simpleFilter": {
        "scope": "AUDIENCE_FILTER_SCOPE_ACROSS_ALL_SESSIONS",
        "filterExpression": {
          "andGroup": {
            "filterExpressions": [
              {
                "orGroup": {
                  "filterExpressions": [
                    {
                      "eventFilter": {
                        "eventName": "experience_impression",
                        "eventParameterFilterExpression": {
                          "andGroup": {
                            "filterExpressions": [
                              {
                                "orGroup": {
                                  "filterExpressions": [
                                    {
                                      "dimensionOrMetricFilter": {
                                        "fieldName": "exp_variant_string",
                                        "stringFilter": {
                                          "matchType": "EXACT",
                                          "value": "ABC-F2948574-3495F49"
                                        },
                                        "atAnyPointInTime": true
                                      }
                                    }
                                  ]
                                }
                              }
                            ]
                          }
                        }
                      }
                    }
                  ]
                }
              }
            ]
          }
        }
      }
    }
  ]
}

The previous code snippet creates the following audience:

• Audience name: "Audience ABC - experiment F2948574 variant 3495F49"
• Audience definition: "Include Users when: events with the name 'experience_impression' have a parameter named 'exp_variant_string' with a value that exactly matches 'ABC-F2948574-3495F49'"
• Membership duration: 30 days

Archive audiences

When an experience ends, you need to archive the audience using the properties.audiences.archive method. Archiving audiences lets you free up space for users to create more experiences. Users will still be able to access data from their archived audiences using historical data in Google Analytics.

Note: Once an audience is archived, its name will no longer be shown in Google Analytics. The Audience ID will replace the name across the various surfaces (such as the dimension picker). We encourage you to show the variant name alongside the audience ID to your end users so users can recognize them in Google Analytics.

Audience targeting

Audience targeting enables your customers to focus an experiment on a group of users who exhibit specific behaviors on their website. For example, a customer can use audience targeting to target high-value customers with special offers and incentives. Through audience targeting, only users in the audience are shown an experiment.

You can enable your customers to set up audience targeting by importing the users in an audience from Google Analytics and then comparing the User IDs or device identifiers from Google Analytics with the identifiers in your system. When these identifiers match, the user is added to the experiment.

To export users from Google Analytics and import them into your client, use the Audience List API. The Audience List API provides a daily snapshot of the users in an audience and lets you export the User IDs or device identifiers in an audience.

We recommend only requesting an Audience List once per day per audience.

Note: Audience targeting is different from the audiences you're creating for the variants of an experiment. Audience targeting enables customers to choose a group of users on their website to target an experiment. The audience variants are the users assigned to one of the variants within an experiment, and they are created only for reporting purposes.

Limitations

When integrating experiment data with Google Analytics, be aware of the following capabilities and constraints.

Pausing experiences

Pausing audiences in Google Analytics is not possible.

To enable users to pause experiences, you must:

• Stop sending the event that triggers when a user becomes a member of a variant
• Ignore any events while the experience is paused
• Add report filters to the Google Analytics report to exclude the paused time

Alternatively, you can provide users with a link to manually change the date in Google Analytics.

Reprocessing experience data

If users are missing event information, Analytics can reprocess their experience data. When Google Analytics re-processes experience data, archived audiences from ended experiences are lost. However, users can still retrieve their experience data through events with exp_variant_string in Explorations. This is less accurate than audiences as events that happened before and after the experience will be present in explorations.

Sampling experience data

If users can be allocated to multiple concurrent audiences, customers expose themselves to cardinality issues (which can cause the "(other)" row to appear). The probability of this happening increases as the concurrent experience count increases and the number of audiences used for other purposes (for example, Ads bidding) increases.

However, if the customer uses Google Analytics 360, Google Analytics will enable automatic expanded data sets when data aggregates under the "(other)" row to give them a more complete picture of their data.

Comparison limits

Google Analytics reports can display up to four comparisons at once. When a user has more than four variants, the user can swap their audiences in comparisons. Consider prioritizing variants with the best conversion rates.

In-house and custom experiments

If you are developing a custom or in-house experiment framework rather than integrating a third-party tool, you can track experiment performance in Google Analytics using client-side events.

To record when a user is exposed to an experiment and variant:

1. Send an event (such as experiment_impression or a custom event) containing parameters identifying the experiment and the variant.
2. Register event-scoped custom dimensions for those parameters in your Google Analytics property (for example, experiment_id and variant_id).
3. For reporting, build custom reports in Explorations and use your custom dimensions to compare metrics across different variants.

Example: gtags

The following example sends a custom event to track an experiment variant impression:

// Triggered when a user is bucketed into a variant
gtag('event', 'experiment_impression', {
  'experiment_id': 'homepage_hero_test',
  'variant_id': 'variant_A'
});

Note: If you use the experience_impression event described in the preceding sections, you can register exp_variant_string as a custom dimension to identify variants.