Decrypt Hyperlocal Targeting Signals

If publishers pass mobile location data to the Ad Exchange that's more specific than a postal code, the Ad Exchange will send a 'hyperlocal' geofence to buyers in a new encrypted field: BidRequest.encrypted_hyperlocal_set.

Timeline

  1. A user installs an ad-supported mobile app and consents to the app accessing and sharing the device location with third parties. This app is also integrated with the Google ads SDK and sends this device location to Google.
  2. Google servers generate a special hyperlocal targeting signal representing a geofence surrounding the device location, such as to protect the user's privacy.
  3. Google servers serialize and encrypt the hyperlocal targeting signal using the security key specific to each buyer. Note that your bidder relies on the same key to decrypt the WINNING_PRICE macro.
  4. Your bidder decrypts and de-serializes the hyperlocal targeting signal into a protocol buffer. Your bidder can then analyze the signal and bid accordingly.

Dependencies

You will need a crypto library that supports SHA-1 HMAC, such as Openssl.

Definition

A hyperlocal targeting signal is defined in the proto like this:

// A hyperlocal targeting location when available.
//
message Hyperlocal {
  // A location on the Earth's surface.
  //
  message Point {
    optional float latitude = 1;
    optional float longitude = 2;
  }

  // The mobile device can be at any point inside the geofence polygon defined
  // by a list of corners.  Currently, the polygon is always a parallelogram
  // with 4 corners.
  repeated Point corners = 1;
}

message HyperlocalSet {
  // This field currently contains at most one hyperlocal polygon.
  repeated Hyperlocal hyperlocal = 1;

  // The approximate geometric center of the geofence area.  It is calculated
  // exclusively based on the geometric shape of the geofence area and in no
  // way indicates the mobile device's actual location within the geofence
  // area. If multiple hyperlocal polygons are specified above then
  // center_point is the geometric center of all hyperlocal polygons.
  optional Hyperlocal.Point center_point = 2;
}

// Hyperlocal targeting signal when available, encrypted as described at
// https://developers.google.com/ad-exchange/rtb/response-guide/decrypt-hyperlocal
optional bytes encrypted_hyperlocal_set = 40;

Each hyperlocal targeting signal contains one or more polygons and a center point. For each polygon, the hyperlocal targeting signal contains:

  • The latitude and longitude of each corner of the polygon sequentially, passed as a repeated corners field.
  • The approximate geometric center of the geofence area, passed in the optional center_point field.

Back to top

Structure of targeting signal

The encrypted hyperlocal targeting signal contained in BidRequest.encrypted_hyperlocal_set contains 3 sections:

  • initialization_vector: 16-bytes.
  • ciphertext: series of 20-byte sections.
  • integrity_signature: 4-bytes.
{initialization_vector (16 bytes)}{ciphertext (20-byte sections)}{integrity_signature (4 bytes)}

The ciphertext byte array is divided into multiple 20-byte sections, with the exception that the very last section may contain between 1 and 20 bytes inclusive. For each section of the original byte_array, the corresponding 20-byte ciphertext is generated as:

<byte_array <xor> HMAC(encryption_key, initialization_vector || counter_bytes)>

Where || is concatenation.

Definitions

Variable Details
initialization_vector 16 bytes - unique to the impression.
encryption_key 32 bytes - provided at account setup.
integrity_key 32 bytes - provided at account setup.
byte_array A serialized HyperlocalSet object, in 20-byte sections.
counter_bytes Byte value showing the ordinal number of the section, see below.
final_message Byte array sent via the BidRequest.encrypted_hyperlocal_set field.
Operators Details
hmac(key, data) SHA-1 HMAC, using key to encrypt data.
a || b string a concatenated with string b.

Calculate counter_bytes

counter_bytes marks the order of each 20-byte section of the ciphertext. Note that the last section may contain between 1 and 20 bytes inclusive. To fill counter_bytes with the correct value when running your hmac() function, count the 20-byte sections (including the remainder) and use the following reference table:

Section number counter_bytes value
0 None
1 … 256 1 byte. The value increments from 0 to 255 sequentially.
257 … 512 2 bytes. The value of the first byte is 0, the value of the second byte increments from 0 to 255 sequentially.
513 … 768 3 bytes. The value of the first two bytes are 0, the value of the last byte increments from 0 to 255 sequentially.

We don't expect the length of BidRequest.encrypted_hyperlocal_set to exceed one kilobyte, even taking further growth into consideration. Nevertheless, counter_bytes can be as long as needed to support a hyperlocal targeting signal of arbitrary length.

Back to top

Encryption scheme

The encryption scheme for hyperlocal targeting signal is based on the same scheme used for decrypting price confirmations.

  1. Serialization: The hyperlocal targeting signal, which is an instance of the HyperlocalSet object as defined in the proto, is first serialized via SerializeAsString() to a byte array.

  2. Encryption: The byte array is then encrypted using a custom encryption scheme designed to minimize size overhead while ensuring adequate security. The encryption scheme uses a keyed HMAC algorithm to generate a secret pad based on the initialization_vector, which is unique to the impression event.

Encryption pseudocode

byte_array = SerializeAsString(HyperlocalSet object)
pad = hmac(encryption_key, initialization_vector || counter_bytes )  // for each 20-byte section of byte_array
ciphertext = pad <xor> byte_array // for each 20-byte section of byte_array
integrity_signature = hmac(integrity_key, byte_array || initialization_vector)  // first 4 bytes
final_message = initialization_vector || ciphertext || integrity_signature

Back to top

Decryption scheme

Your decryption code must 1) decrypt the hyperlocal targeting signal using the encryption key, and 2) verify the integrity bits with the integrity key. The keys will be provided to you during account setup. There aren't any restrictions on how you structure your implementation. For the most part, you should be able to take the sample code and adapt it according to your needs.

  1. Generate your pad: HMAC(encryption_key, initialization_vector || counter_bytes)
  2. XOR: Take this result and <xor> with the ciphertext to reverse the encryption.
  3. Verify: The integrity signature passes 4 bytes of HMAC(integrity_key, byte_array || initialization_vector)

Decryption pseudocode

(initialization_vector, ciphertext, integrity_signature) = final_message // split up according to length rules
pad = hmac(encryption_key, initialization_vector || counter_bytes)  // for each 20-byte section of ciphertext
byte_array = ciphertext <xor> pad // for each 20-byte section of ciphertext
confirmation_signature = hmac(integrity_key, byte_array || initialization_vector)
success = (confirmation_signature == integrity_signature)

Sample C++ code

Included here is a key function from our complete decryption example code.

bool DecryptByteArray(
    const string& ciphertext, const string& encryption_key,
    const string& integrity_key, string* cleartext) {
  // Step 1. find the length of initialization vector and clear text.
  const int cleartext_length =
      ciphertext.size() - kInitializationVectorSize - kSignatureSize;
  if (cleartext_length < 0) {
    // The length cannot be correct.
    return false;
  }

  string iv(ciphertext, 0, kInitializationVectorSize);

  // Step 2. recover clear text
  cleartext->resize(cleartext_length, '\0');
  const char* ciphertext_begin = string_as_array(ciphertext) + iv.size();
  const char* const ciphertext_end = ciphertext_begin + cleartext->size();
  string::iterator cleartext_begin = cleartext->begin();

  bool add_iv_counter_byte = true;
  while (ciphertext_begin < ciphertext_end) {
    uint32 pad_size = kHashOutputSize;
    uchar encryption_pad[kHashOutputSize];

    if (!HMAC(EVP_sha1(), string_as_array(encryption_key),
              encryption_key.length(), (uchar*)string_as_array(iv),
              iv.size(), encryption_pad, &pad_size)) {
      printf("Error: encryption HMAC failed.\n");
      return false;
    }

    for (int i = 0;
         i < kBlockSize && ciphertext_begin < ciphertext_end;
         ++i, ++cleartext_begin, ++ciphertext_begin) {
      *cleartext_begin = *ciphertext_begin ^ encryption_pad[i];
    }

    if (!add_iv_counter_byte) {
      char& last_byte = *iv.rbegin();
      ++last_byte;
      if (last_byte == '\0') {
        add_iv_counter_byte = true;
      }
    }

    if (add_iv_counter_byte) {
      add_iv_counter_byte = false;
      iv.push_back('\0');
    }
  }
}

Sample hyperlocal signal and keys

To test and verify your code:

  1. Convert a string containing 308 hex characters into an array of 154 bytes. For example, given the following string:
    E2014EA201246E6F6E636520736F7572636501414243C0ADF6B9B6AC17DA218FB50331EDB376701309CAAA01246E6F6E636520736F7572636501414243C09ED4ECF2DB7143A9341FDEFD125D96844E25C3C202466E6F6E636520736F7572636502414243517C16BAFADCFAB841DE3A8C617B2F20A1FB7F9EA3A3600256D68151C093C793B0116DB3D0B8BE9709304134EC9235A026844F276797
    
    convert it into a 154-byte array as follows:
    const char serialized_result[154] = { 0xE2, 0x01, 0x4E, ... };
    
  2. Call the BidRequest.ParsePartialFromString() method to deserialize the 154-byte array into a BidRequest protocol buffer.
    BidRequest bid_req;
    bid_req.ParsePartialFromString(serialzed_result);
    
  3. Verify that the BidRequest has only 3 fields:
    • encrypted_hyperlocal_set
      Declared in the BidReqeust message.
    • encrypted_advertising_id
      Declared in the BidReqeust.Mobile message.
    • encrypted_hashed_idfa
      Declared in the BidReqeust.Mobile message.

    For example:

    encrypted_hyperlocal_set:(
        {  100,  100 },
        {  200, -300 },
        { -400,  500 },
        { -600, -700 },)
    encrypted_advertising_id: { 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11 }
    encrypted_hashed_idfa : { 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF, 0xF1 }
    
  4. Use the following encryption_key and integrity_key to decrypt the 3 fields and verify that you decrypt them correctly.
    encryption_key = {0x02, 0xEE, 0xa8, 0x3c, 0x6c, 0x12, 0x11, 0xe1, 0x0b,
        0x9f, 0x88, 0x96, 0x6c, 0xee, 0xc3, 0x49, 0x08, 0xeb, 0x94, 0x6f, 0x7e,
        0xd6, 0xe4, 0x41, 0xaf, 0x42, 0xb3, 0xc0, 0xf3, 0x21, 0x81, 0x40};
    
    integrity_key = {0xbf, 0xFF, 0xec, 0x55, 0xc3, 0x01, 0x30, 0xc1, 0xd8,
        0xcd, 0x18, 0x62, 0xed, 0x2a, 0x4c, 0xd2, 0xc7, 0x6a, 0xc3, 0x3b, 0xc0,
        0xc4, 0xce, 0x8a, 0x3d, 0x3b, 0xbd, 0x3a, 0xd5, 0x68, 0x77, 0x92};
    

Back to top

Detect stale response attacks

To detect stale response attacks, we recommended filtering responses with a timestamp that differs significantly from the system time, after accounting for timezone differences. Our servers are set to PST/PDT time.

For implementation details, see “Detecting Stale Response Attacks” in the Decrypting Price Confirmations article.

Back to top

Java library

Instead of implementing the crypto algorithms to encode and decode the hyperlocal targeting signals, you can use DoubleClickCrypto.java. For more information, see Cryptography.

Send feedback about...

Real-Time Bidding Protocol
Real-Time Bidding Protocol