Bidding and Auction services integration and optimization

The Bidding and Auction Services for Android design proposal details the execution and data flow of running auctions on Android using the Trusted Bidding and Auction server. To ensure data in transit is handled only by privacy-preserving APIs and trusted servers, data is encrypted between the client and server using bidirectional Hybrid Public Key Encryption.

Illustration of the protected audience flow. Three columns represent how data moves between devices, untrusted seller services, and a trusted execution environment. — The Protected Audience auction flow.

To run the auction as detailed earlier, the seller ad tech on the device must perform the following steps:

Gather and encrypt data for server auction
Send a request to an Untrusted Seller Service
Receive a response from an Untrusted Seller Service
Decrypt the Protected Audience auction response and get the auction result

Protected Audience is introducing two new APIs in order to support running server auctions:

The getAdSelectionData API gathers data for the server auction and generates an encrypted payload containing auction data. The Bidding and Auction server uses this payload to run an auction, generate the auction result, and return the auction result.
On-device ad tech clients can call the persistAdSelectionResult API to decrypt the result generated by the server auction and get the winning ad render URL.

The seller ad tech on the device needs to integrate and build the following to run an auction:

Gather and encrypt data for server auction Seller: The ad tech should call getAdSelectionData API to get the encrypted payload.
Send request to Untrusted Seller Service Send: An HTTP POST or PUT request containing the encrypted payload generated by getAdSelectionData API to their untrusted seller service and data required by the untrusted seller service to generate contextual results.
Receive response from Untrusted Seller Service: Response from untrusted seller service would contain the encrypted protected audience auction result and the contextual auction result.
Decrypt protected audience auction response and get the auction result: To decrypt the protected audience auction result, seller ad tech should call the persistAdSelectionResult API. The outcome generated by persistAdSelectionResult will help ad techs to determine whether contextual ad or protected audience ad won the auction and the URI of the winning protected audience ad if applicable.

Features supported for server auction

We aim to support all features currently available for on-device auction. The timeline for supporting these features in server auction is as follows:

	On-device Auction		Server Auction
	Developer Preview	Beta	Developer Preview	Beta
Event-level win reporting	Q1 '23	Q3 '23	N/A	Q4 '23
Waterfall mediation	Q1 '23	Q4 '23	N/A	Q1 24
Frequency cap filtering	Q2 '23	Q3 '23	N/A	Q4 '23
Pass contextual ads to ad selection workflow for filtering	Q2 '23	Q1 '24	N/A	N/A
Interaction reporting	Q2 '23	Q3 '23	N/A	Q4 '23
Join custom audience delegation	Q3 '23	Q4 '23	N/A	Q4 '23
non-CPM billing	Q3 '23	Q4 '23
Debug reporting	Q3 '23	Q4 '23	Q3 '23	Q4 '23
Open Bidding Mediation	N/A	N/A	N/A	Q1 '24
App install ads filtering	Q2 '23	Q1 '24	N/A	Q1 '24
Currency management	N/A	N/A	N/A	Q1 '24
K-anon integration	N/A	Q1 '24	N/A	Q1 '24
Private Aggregation integration	N/A	N/A	N/A	Q3 '24

Run Server Auctions using Protected Audience APIs

In the Developer Preview track, AdSelectionManager exposes two new APIs: getAdSelectionData and persistAdSelectionResult. These APIs allow ad tech SDKs to integrate with Bidding and Auction servers.

Gather and encrypt data for a server auction

The getAdSelectionData API generates the required input for Bidding and Auction components such as BuyerInput and ProtectedAudienceInput, and encrypts the data before making the result available to the caller. To prevent data leaking across apps, this data contains information from all buyers present on the device. Read more about this decision in the privacy considerations section and strategies to optimize this in the size considerations section.

To access the API, access to Protected Audience API must be enabled, and the ACCESS_ADSERVICES_CUSTOM_AUDIENCE permission must be defined in the caller's manifest.

public class AdSelectionManager {
    public void getAdSelectionData(
            GetAdSelectionDataRequest getAdSelectionDataRequest,
            Executor executor,
            OutcomeReceiver<GetAdSelectionDataOutcome, Exception> receiver) {}
}

GetAdSelectionDataRequest

The caller must set the seller field in the request as it is used to run enrollment checks before servicing the request.

public class GetAdSelectionDataRequest {
  Public setSeller(AdTechIdentifier seller);
}

After the request is validated, on-device buyer data is composed into BuyerInput and ProtectedAudienceInput. The final payload object is then encrypted using bidirectional Hybrid Public Key Encryption.

GetAdSelectionDataOutcome

GetAdSelectionDataOutcome is generated as the outcome of getAdSelectionData API. It contains the following:

adSelectionId: an opaque integer to identify this invocation of getAdSelectionData. The ad tech client should persist this adSelectionId value because it acts as the pointer to the getAdSelectionData call. This identifier is required by the persistAdSelectionResult API to decrypt the auction result from Bidding and Auction server and is also required by the reportImpression and reportEvent APIs.
adSelectionData: This is the encrypted auction data which would be required by Bidding and Auction server to run auctions. This method contains:
1. Filtered Custom Audience data based on frequency capping, app install filters and server auction requirements for Custom Audiences.
2. In a future version, it will contain app install data.

public class GetAdSelectionDataOutcome {
  Public getAdSelectionId(long adSelectionId);
  public byte[] getAdSelectionData();
}

Errors, exceptions and failure handling

If the ad selection data generation can't be completed successfully due to reasons such as invalid arguments, timeouts, or excessive resource consumption, the OutcomeReceiver.onError() callback provides an AdServicesException with the following behaviors:

If getAdSelectionData is initiated with invalid arguments, the AdServicesException` indicates an IllegalArgumentException as the cause.
All other errors receive an AdServicesException with an IllegalStateException as the cause.

Send a request to an untrusted seller service

Using AdSelectionData, the on-device SDK can send a request to their seller's ad service by including the data in a POST or PUT request:

fetch('https://www.example-ssp.com/auction', {
  method: "PUT",
  body: data,
...
})

The on-device SDK is responsible for encoding this data. It is recommended to use a space efficient solution such as sending the request to the seller's ad service as multipart/form-data.

Receive a response from an untrusted seller service

As detailed in the Bidding and Auction Server explainer, when the untrusted seller service receives the request, it makes calls to partner buyers for contextual ads.

The untrusted seller service forwards the encrypted adSelectionData and AuctionConfig to the Bidding and Auction server's SellerFrontEnd service running in a TEE.

When the Protected Audience auction is complete, the SellerFrontEnd service encrypts the auction result and returns it as a response to the untrusted seller service.

The untrusted seller service sends a response to the device containing contextual ad and / or encrypted Protected Audience auction result.

On receiving the response, the seller ad tech code on device could choose to only use the contextual ad in the response or if it deems that there is incremental value in getting the Protected Audience result, it can choose to decrypt the Protected Audience result by calling the PersistAdSelectionResult API.

PersistAdSelectionResult API

To decrypt the Protected Audience result, the seller ad tech can call the second Protected Audience API persistAdSelectionResult. The API decrypts the result and returns an AdSelectionOutcome, the same object that is returned from an on-device auction today.

To access the API, the caller must enable access to Protected Audience API and define the ACCESS_ADSERVICES_CUSTOM_AUDIENCE permission in their manifest.

    public void persistAdSelectionResult(
            PersistAdSelectionResultRequest persistAdSelectionResultRequest,
            Executor executor,
            OutcomeReceiver<AdSelectionOutcome, Exception> receiver) {}

PersistAdSelectionResultRequest

The caller must set the following in the request:

public final class PersistAdSelectionResultRequest {
  Public setAdSelectionId(long adSelectionId);
  public setSeller(AdTechIdentifier seller);
  public setAdSelectionResult(byte[] adSelectionResult);
}

adSelectionId: The opaque identifier generated by the getAdSelectionData call whose result the caller wants to decrypt.
seller: The seller ad tech identifier must be set in the request to run enrollment checks before servicing the request.
adSelectionResult: The encrypted auction result generated by the Bidding and Auction server that the caller wants to decrypt.

AdSelectionOutcome response

If there is a Protected Audience winner, then AdSelectionOutcome returns the winning ad render URI.Once the adSelectionResult is decrypted, the reporting data is persisted internally. The OutcomeReceiver.onResult() callback returns an AdSelectionOutcome that contains:

URI: If there is a winning Protected Audience ad, then an ad render URL for the winning ad is returned. If there is no Protected Audience winner, then `Uri.EMPTY is returned.
adSelectionId: The adSelectionId associated with this server auction run.

Errors, exceptions and failure handling

If getAdSelectionData is initiated with invalid arguments, the AdServicesException indicates an IllegalArgumentException as the cause.
All other errors receive an AdServicesException with an IllegalStateException as the cause.

Privacy Considerations

adSelectionData is encrypted to ensure that data in transit is accessible only to PPAPI and the trusted servers.

Despite encryption, data leakage can occur due to adSelectionData size. The adSelectionData size can vary due to:

Changes in CustomAudience data present on device.
Changes to CustomAudience filtering logic.
Changes to input to getAdSelectionData call.

Change in adSelectionData size can be used for generating a cross-app identifier as mentioned in the 1-bit leak discussion. Many mitigations applicable to 1-bit leak are also applicable here.

To manage these leaks, we plan to generate the same adSelectionData for all calls to getAdSelectionData API. In initial releases, all the CustomAudiences on the device are used for creating adSelectionData and the encrypted payload will be padded to mask size variations. We will also restrict the influence of GetAdSelectionData input parameters on the adSelectionData generated.

However, generating the same adSelectionData for all ad techs using all the on-device auction data creates a large payload that now needs to be transferred in every call to the ad tech server. Using all the on-device custom audiences to generate auction payload also opens the ecosystem to abuse from malicious entities. We've covered these concerns in the Size optimizations and Abuse mitigations sections below.

Size optimizations

The ad tech client SDK is expected to package the encrypted bytes of adSelectionData into the HTTP PUT/POST contextual call made to the ad tech server. For lower round-trip time latency and cost, it is necessary to reduce the adSelectionData size as much as possible while not impacting utility.

We plan to explore and potentially introduce the following optimizations in the upcoming releases to reduce the adSelectionData size:

Payload generated in a fixed set of bucket sizes with padding: To minimize the leakage from size variations while still allowing for lower payloads, we suggest using fixed size bucketing for the generated payload. Keeping the number of buckets small, for instance, 7 will lead to less than 3 bits of leaked entropy per call to getAdSelectionData.

If on-device data exceeds the maximum bucket size, then strategies mentioned below such as priority values would be used to decide which data gets dropped.
Buyer Configuration: We're assessing the feasibility of letting buyers set up a per-buyer payload configuration. This configuration would be useful for identifying which auctions a buyer is interested to join. If feasible, during enrollment, a buyer ad tech could register an endpoint from which Protected Audience would fetch payload configuration at a daily regular cadence. Alternatively, privacy-preserving APIs would expose an API to allow buyer ad techs to register this endpoint.

This configuration would then be used to assess a buyer's contribution to adSelectionData generated for each getAdSelectionData request.

The buyer payload configuration would allow buyers to specify:
1. Allowed sellers list: Buyer CustomAudiences will be added to the payload only if the getAdSelectionData call is initiated by a seller in the allowlist. We would fetch the payload configuration at a daily cadence to keep the allowlist up to date.
2. Per-seller size limit: Buyer could specify a per-seller size limit to determine the data size to be sent in the payload when an auction is initiated by a certain seller. This would be useful if a buyer wants to devote more resources to processing auction data from certain sellers. The SellerFrontendService forwards only buyer-specific data to each BuyerFrontendService. So, by defining a per-seller size limit, a buyer could explicitly control the amount of data ingested and processed by their Bidding and Auction server's BuyerFrontendService for auctions run by a seller.
Seller Configuration: We're assessing the feasibility of a per-seller auction configuration that would allow sellers to define auction parameters to control payload size and auction participants. If feasible, during enrollment, the seller ad tech would be able to specify the endpoint from where Protected Audience could fetch the per-seller auction configuration at a regular cadence. This configuration would then be used to determine the composition and limits of adSelectionData generated for each getAdSelectionData request.

Similar to the buyer configuration, a per-seller configuration would allow sellers to specify which set of buyers they expect to see in an auction and to specify limits on per buyer contribution to payload size.

The seller auction configuration would allow sellers to specify:
1. Allowed buyer list: For auctions initiated by the given seller, only the buyers in the allowlist would be able to contribute CustomAudiences for the auction. The auction configuration would need to be updated daily to keep the allowlist up to date with server-side buyer allowlist.
2. Per-buyer size limit: Sellers could specify a per-buyer limit to regulate the data size uploaded by each buyer into the payload being sent to SellerFrontendService. If the buyer exceeds the per-buyer size limit, the CustomAudience priority set in buyer payload configuration would be used to get the data in the expected limits.
3. Per-buyer priority: Allow sellers to set per-buyer priority. Buyer priority would be used to identify which buyer data should be kept in the payload if the payload size exceeds the payload size limit.
4. Max size limit for the payload: Different sellers might have different resource allocation and might want to set a max size limit for the per-request auction payload. The max size limit would respect the fixed size buckets set by the Protected Audience API.
Custom Audience changes
1. Specify Custom Audience priority: Allow buyers to specify a priority value in a Custom Audience. The priority field would be used to identify custom audiences which should be included in an auction if the set of buyer custom audiences exceed the per-seller or per-buyer size limits. An unspecified priority value in a Custom Audience would default to 0.0.
Payload Data Changes
1. Reduce data sent in the payload: As detailed in Bidding and Auction services payload optimization, higher payload is driven by custom audience ads data, user bidding signals, Android signals. Higher payloads could be lowered by:
  1. Having the client send ad render IDs (instead of ad objects) in the payload.
  2. Having the client send no ad data in the payload.
  3. Not sending user bidding signals in the client payload.

While strategies mentioned above allow ad techs to define configurations to manage adSelectionData payload composition and limits, they could also become a factor for modulating adSelectionData size by changing configuration parameters. To avoid this, the configuration would be fetched daily by Protected Audience from the configured endpoint.

Latency optimization

For server auctions to have a desirable level of utility, we need to ensure that getAdSelectionData API and persistAdSelectionResult API have low latency per call. While we aim to deliver feature support for APIs in 2023, our subsequent release will focus on latency benchmarks and optimizations for the APIs.

We are exploring the following strategies to keep the latency within acceptable limits:

Pre-generation of Protected Audience data per seller: Since seller auction configuration and buyer payload configuration will be stable for a considerable duration (daily), the platform could pre-compute and store the eligible Protected Audience data.

This would require the platform to build a mechanism to monitor custom audience updates and modify the pre-generated Protected Audience data based on the updates. The platform would also need to declare SLOs on the race delay ad tech could expect between custom audience updates and seeing the change in the adSelectionData` generated for the server auction.

Since a device provides a limited resource computation model with varying process priorities, we recognize that providing this pre-generation facility must come with high reliability and SLOs guarantees.

Pre-generating the Protected Audience data would then be based on
1. Seller opt-in to pre-generate Protected Audience data.
2. Buyers eligible to participate in an auction initiated by a particular seller.
3. Identifying custom audiences per-buyer which would be part of the payload based on:
  1. Per-buyer size limits, per-buyer priority and max size limits defined in seller configuration,
  2. Per-seller size limit, custom audience priority defined in buyer configuration.
Eager application of negative filtering: If preferred by a seller, the platform could pre-compute the adSelectionData by pre-generating Protected Audience data and applying negative filtering off the critical getAdSelectionData call. This would allow sellers to balance lowering latency while accepting staleness in negative filtering.

The platform could provide this support by providing a default option in the Seller configuration with a staleness limit and an override option in getAdSelectionData to allow for freshest computation if required. Alternatively, the platform could provide an additional initialization API to be called before getAdSelectionData for warming up the auction.
Payload computation for multiple auctions: In certain scenarios, it might be preferable to have a latency-performant API at the cost of increased data staleness. To provide this, the platform could introduce an initialization API to compute the entire payload and provide a reference to the computed payload to the caller.

For subsequent calls to getAdSelectionData, the caller could provide reference to the pre-computed payload to be used for adSelectionData generation.

All the three strategies mentioned above are in the initial exploration stage and meant to describe the direction the platform might be taking to optimize for latency. As we explore more detailed latency profiles of the API and ad tech requirements, we'll continue proposing additional strategies.

Abuse mitigation and identification

As mentioned in Privacy considerations, adSelectionData is generated by using all the buyer data on the device.

However, if all buyer data on the device is used to generate the adSelectionData output, then a malicious entity could pose as a buyer and create fraudulent buyer data to degrade Android performance, bloat payload to increase cost for an ad tech to run auctions or run bidding, and so forth.

Mitigation

Some measures mentioned in the size considerations section such as buyer payload configuration containing allowlisted sellers and seller auction configuration containing allowlisted buyers would help to exclude unexpected data in the payload.

Other size consideration measures such as allowing SSPs to specify buyer priority, placing per-buyer quota in the generated payload, and setting a max size per auction payload can also help mitigate the impact of malicious payload bloating. These measures are intended to allow ad techs to define which ad tech they collaborate with, and to set acceptable limits on the payload they would need to process.

As mentioned earlier, all mitigations introduced for anti-abuse and size restrictions must adhere to privacy considerations.

Identification of malicious entities

While mitigations mentioned above protect the adSelectionData generation for server auctions, they do not help identify malicious entities or to protect the platform from abuse such as the creation of an unprecedented number of custom audiences from a buyer.

To ensure platform stability and health, we need to find a mechanism to identify malicious entities, to identify abuse vectors, and to identify the motivation for the specific attacks. In later releases, we'll introduce explainers detailing the potential abuse vectors and protections in place to counter them.