App Engine's Traffic Splitting tool allows you to roll out features for your app slowly over a period of time, similar to what Google does when rolling out a new feature over a few days or weeks. Traffic Splitting also allows you to do A/B Testing. Traffic Splitting works by splitting incoming requests to different versions of your app.
To set up Traffic Splitting, choose a non-default version of your app with code you want to test, specify the percentage of traffic it should receive, and choose the type of splitting to use. It also important to pay attention to the effects of caching on the static and dynamic resources in your app.
Adding and Configuring Traffic Splitting
You configure Traffic Splitting on the Versions tab in the Admin Console, as pictured on the right:
Note: You cannot enable Traffic Splitting on the default version of your app. Make sure you have at least two major versions of your app available, so that you have one non-default version.
Traffic can be sent to different versions of your app using either an IP address or a cookie, as described in the next section.
The Types of Splitting: IP Address and Cookie
You may choose only one method for splitting traffic per version of your app:
- IP address. Users are routed to the app version based on their IP address.
- Cookie. Users are routed to the app version based on an HTTP cookie.
IP address splitting is easier to set up, but delivers less precise and detailed statistics. For example, if you ask for 5% of traffic to be delivered to an alternate version, you may get a range of 3–7%. Because it is the easiest to use, IP splitting is the default setting in the Administration Console.
Cookie splitting gives you finer control, and more accurate traffic splitting. However, it is more difficult to set up than IP address-based traffic splitting.
Splitting Traffic Using IP Address
IP splitting requires less effort to manage, because it is based simply on the user's IP address. When the application receives an address, it hashes the IP address of that request to a value between 0–999. This value determines which version a user is routed to.
IP address splitting has some significant limitations:
- Randomization Effects. Because IP addresses are independently assigned to versions, the resulting traffic split will differ somewhat from what you specify. However, the more traffic that your application receives, the more accurate the split will be, due to the Law of Large Numbers.
- IP addresses change. IP addresses are reasonably sticky, but are not permanent. Users connecting from cell phones may have a shifting IP address throughout a single session. Similarly, a user on a laptop may be moving from home to a cafe to work, and hence also shifting through IP addresses. As a result, the user may have an inconsistent experience with your app as their IP address changes.
- Internal requests don't work. Requests sent to your app from outside of Google's cloud infrastructure will work normally. However, requests sent from your app to another app (or to itself) are not sent to a version because they all originate from the a small number of IP addresses. This applies to any traffic between apps within Google's cloud infrastructure. If you need to send requests between apps, use cookie splitting instead.
Splitting Traffic Using Cookies
Cookie splitting works by using HTTP cookies to assign users a value between 0–999. This approach makes it much easier to get an accurate assignment of users to versions, and hence we allow smaller slices of traffic (as small as 0.1%). Cookie splitting also has some limitations:
- Cookie management is required for mobile or desktop apps. If you are writing a mobile app or a desktop client of some kind, then it will need to manage cookies. When App Engine sends back a
Set-Cookieheader, you need to store this cookie and send it with each subsequent request. Browser-based apps work fine, as browsers automatically manage cookies.
- Internal requests require extra work. Requests sent from your app server-side to another app (or to itself) can be sent into a version, but doing so requires you to forward the user's cookie as part of the request. However, internal requests that don't originate from the user are not recommended for versions.
Setting the Cookie
The cookie is set for you automatically. The cookie App Engine uses for traffic splitting is
GOOGAPPUID, and its range of valid values is 0–999. This cookie is set to a random valid value if and only if the following are true:
- Your app is configured to use cookie splitting.
- The HTTP request from the user does not already contain the
The response from your app does not already contain a
Set-Cookie: GOOGAPPUID=...header. This allows your app to control which version a user gets.
Each request arriving at an app is sent to a particular version of the app. Normally, the versions are distinguished by URL. For example, consider an app called
codeninja with three active major versions:
default. All traffic sent to
http://codeninja.appspot.com goes to the
default version, but you can send traffic to other versions by including their version name as a prefix (for example, you could access the beta version of the codeninja app via
Let's say you are rolling out
beta slowly to all your users, and it is currently set to receive 20% of all traffic. App Engine automatically assigns 20% of available values (0–199) to the
beta version. Depending on which traffic splitting method you choose, when an end user arrives at your app, App Engine either hashes their IP or examines their cookie. If the result of that is in the range 0–199, then the end user is sent to the
At the same time, you also have a highly experimental version (
alpha) which is receiving 1% of traffic, to see how it influences user behavior. App Engine assigns this version 1% of possible values guaranteed not to overlap with
beta. For example, maybe this range is 990–999.
In this example, users are sent to a version based on their IP or hashed cookie as follows:
- 0–199: users sent to
- 200–989: users sent to
- 990–999: users sent to
How Caching Impacts Traffic Splitting
When using Traffic Splitting, you need to treat cached resources carefully. The cached versions of static and dynamic resources can create an inconsistent user experience or unanticipated application behavior between versions. Caching issues exist for all apps, but traffic splitting will make subtle flaws in caching much more apparent.
You can avoid this problem for dynamic resources by setting the
headers. These headers indicate to proxies that the resource is dynamic. It is
best to set both as not all proxy servers properly support the HTTP/1.1
Cache-Control header. If you want more information on caching in general, try these web pages:
For cacheable static resources that vary between versions, you can simply change the URL for cacheable static resources that vary between versions. If each version of a static resource is serving from different URLs, then those resources can happily co-exist in proxy servers and browser caches.
If the app sets the
Vary: Cookie header, the uniqueness of a resource is computed by combining the cookies and the URL for the request. This approach increases the burden on cache servers. There are 1000 possible values of
GOOGAPPUID, and hence 1000 possible entries for each URL for your app. Depending on the load on the proxies between your users and your app, this may decrease cache hit rate. Also note that for the 24 hours after adding a new batch of users to a version, they may still see cached resources. However, using
Vary: Cookie can make it easier to rename static resources that are changing between versions.
Vary: Cookie technique doesn't work in all circumstances. In general, if your app is using cookies for other purposes, then you must consider how this affects the burden on proxy servers. If
codeninja had its own cookie that had 100 possible values, then the space of all possible cache entries becomes
100 * 1000 = 100,000. That's very big number. In the worst case, there is a unique cookie for every user. Two common examples of this are Google Analytics (
__utma) and SiteCatalyst (
s_vi). In this case, every user gets a unique copy, which severely degrades cache performance and also may increase the number of billable instance hours consumed by your app.