Exchange Connectors

Although Open Bidder only provides out-of-the-box support for DoubleClick Ad Exchange, the framework is exchange-neutral. In this guide we will show you how to add support to Open Bidder for handling requests from other exchanges that use a custom/arbitrary protocol. (See also how to support an OpenRTB exchange.)

Create the Exchange class

"Step zero" is very simple: your exchange needs to extend the Platform class. Your subclass will have a single instance that's used to bind objects like requests, responses and request receivers to the exchange.

public final class MyExchange extends Exchange {
  public static final Exchange INSTANCE = new MyExchange();

  private MyExchange() {
    super("MyExchange");
  }

  @Override public Object newNativeResponse() {
    return new StringBuilder();
  }
}

Implement the exchange's bid request receiver

The first step of extending Open Bidder for another exchange is to implement a RequestReceiver subclass, specific to that exchange. This class has three goals:

  1. Map the HTTP request to an Open Bidder BidRequest.
  2. Invoke the BidController that runs the interceptor stack on that request.
  3. Map the Open Bidder BidResponse to an HTTP response sent to the exchange.

Below is an example of a receiver for bid requests:

@Singleton public class MyExchangeRequestReceiver
extends RequestReceiver<BidController> {

  @Inject public MyExchangeRequestReceiver(
      MetricRegistry metricsRegistry,
      BidController controller) {
    super(MyExchange.INSTANCE, metricsRegistry, controller);
  }

  @Override public void receive(HttpReceiverContext ctx) {
    try {
      // Step 1. Map the HTTP request to an Open Bidder bid request
      String nativeRequest = ObHttpUtils.readContentString(ctx.httpRequest());
      BidRequest request = BidRequest.newBuilder()
          .setExchange(MyExchange.INSTANCE)
          .setHttpRequest(ctx.httpRequest())
          .setNativeRequest(nativeRequest)
          .build();
      setRequestId(nativeRequest.split(";")[0]);

      // Step 2. Execute the Open Bidder interceptor stack on the bid request
      BidResponse response = BidResponse.newBuilder()
          .setExchange(MyExchange.INSTANCE)
          .setHttpResponse(ctx.httpResponse())
          .build();
      controller().onRequest(request, response);

      // Step 3. Map the Open Bidder BidResponse back to the exchange's format.
      ctx.httpResponse().printContent(((StringBuilder) response.nativeResponse()).toString());

      successResponseMeter().mark();
    } finally {
      clearRequestId();
    }
  }
}

Most of the time the receiver is very simple; besides those essential three steps, you may need some monitoring (the base class defines several standard metrics), logging and error handling. The receiver is also a general extension point that you can use to handle any transport-layer requirements or customize what happens around invocations to the controller.

We will now break down each of the steps in more detail.

Map the HTTP request to an Open Bidder BidRequest

This example is for a non-OpenRTB exchange, so we just extract the request data from the HTTP message and stuff it in the BidRequest. The native request for the sample exchange is just a string with some semicolon-separated fields, and we copy that string as-is into BidRequest.nativeRequest, but this could be more elaborate; for example you could parse that message into model objects.

Execute the Open Bidder interceptor stack on the BidRequest

When the handler invokes the controller, the corresponding interceptor stack will be executed, populating a response which in this case will contain the bids. Your exchange's receiver can access the singleton BidController via injection. Read the Open Bidder Guice guide for more information on using and dependency injection in Open Bidder.

Map the Open Bidder BidResponse back to the exchange-specific response format

Again this step is trivial here; this exchange's bid response is also a semicolon-separated string that we take from the BidResponse and dump in the HTTP response. Notice that we used StringBuilder as the type of BidResponse.nativeResponse: it's usually a good idea to use a mutable type or data structure there, so complex bidding logic may populate response data incrementally, e.g., in a loop that would generate bids for multiple impressions.

In both cases, besides invoking the mapper and wrapping/unwrapping data through HTTP requests and responses, the receiver may need to handle other HTTP properties: for example, some exchange might require setting specific response headers or status codes. Our example needs none of that.

Bind exchange receiver

After writing the exchange's receiver, we must configure it with Guice. This step follows the Guice guide's instructions that suggest you create an AbstractModule to bind your receiver. The following code also illustrates how to support command-line parameters, in this case the request endpoint, via Open Bidder's integration to the JCommander library.

@Parameters(separators = "=")
public class MyExchangeModule extends AbstractModule {
  @Parameter(names = "--my_bid_path",
      description = "Path spec for MyExchange's bid requests")
  private String path = "/bid_request/my";

  @Override protected void configure() {
    bind(String.class).annotatedWith(BidRequestPath.class).toInstance(path);
    Multibinder.newSetBinder(binder(), HttpRoute.class).addBinding()
        .toProvider(BidRouteProvider.class).in(Scopes.SINGLETON);
  }

  private static class BidRouteProvider extends AbstractHttpRouteProvider {
    @Inject private BidRouteProvider(
        @BidRequestPath String path, MyExchangeRequestReceiver receiver) {
      super(HttpRoute.get("bid", path, receiver, Feature.BID));
    }
  }

  @BindingAnnotation @Target({ FIELD, PARAMETER, METHOD }) @Retention(RUNTIME)
  public static @interface BidRequestPath {}
}

In order for the module to be applied, you need to create a customized bidder server project that explicitly adds it. See Using Guice to Add Request Receivers.

Now when a bidder starts it will send requests to /bid_request/my, the receiver we created for this endpoint.

We're done writing code! Open Bidder has been set up to receive requests from another exchange and execute the same interceptor stack that will be used to process DoubleClick Ad Exchange requests. Now you should go back to your interceptor code and make sure that any code that was specific to the DoubleClick bid request type will handle requests from this new exchange.

Testing

You can unit test receivers easily, without need to run a full bidder:

@Test public void testHttpRequest() throws IOException {
  HttpRequest httpRequest = StandardHttpRequest.newBuilder()
      .setProtocol(Protocol.HTTP_1_1)
      .setMethod("POST")
      .setUri("http://example.com")
      .setLocalAddress(InetSocketAddress.createUnresolved("localhost", 8080))
      .printContent("1;300;250")  // request-id;width;height
      .build();
  HttpResponse.Builder httpResponseBuilder = StandardHttpResponse.newBuilder();
  MyExchangeRequestReceiver receiver = new MyExchangeRequestReceiver(
      new MetricRegistry(),
      BiddingTestUtil.newBidController(new SimpleBidInterceptor()));
  receiver.receive(new DefaultHttpReceiverContext(httpRequest, httpResponseBuilder));
  HttpResponse httpResponse = httpResponseBuilder.build();
  assertThat(httpResponse.getStatusCode()).isEqualTo(HttpStatus.SC_OK);
  assertThat(httpResponse.contentReader().readLine()).isEqualTo("1;1.0;html-or-vast;");
}

You may have already seen other unit tests for interceptors; they are typically "higher-level", it is easier to just create the BidRequest and BidResponse objects and invoking a BidController. But this unit test takes a "lower-level" approach, using HttpRequest as input, invoking an exchange's RequestReceiver, and inspecting the output from an HttpResponse. This lower-level approach needs extra code, and it is only really necessary for testing receivers; but you may also find it desirable in other circumstances, for example to create tests that use as test data real-world HTTP messages captured from bidder traffic.

@Test public void testModule() {
  List<Module> modules = asList(
      binder -> {
        // Prevents any actual network access by tests
        binder.bind(HttpTransport.class).toInstance(ResourceHttpTransport.create());
        binder.bind(Integer.class).annotatedWith(BidderHttpPort.class)
            .toInstance(BidderHttpPort.DEFAULT);
        binder.bind(String.class).annotatedWith(CallbackHost.class).toInstance("mybidder.com");
        binder.bind(boolean.class).annotatedWith(ExtendedMacros.class).toInstance(false);
      },
      new StandardSnippetProcessorModule(),
      new BidModule(),
      new MyExchangeModule());
  JCommander jcommander = new JCommander(modules);
  jcommander.parse("--my_bid_path=/mybidrequests");
  assertThat(Guice.createInjector(Stage.DEVELOPMENT, modules)).isNotNull();
}

Finally, it's a good idea to also create a unit test for the whole module. This test allows you to make sure that any dependency injections required by your exchange module are being satisfied, either by bindings created by this module or by some required dependencies.

Bidding on multiple exchanges

At this time, Open Bidder supports pluggable exchanges, but you can only plug in a single exchange in any particular bidder. You can bid on multiple exchanges from a single Open Bidder project, but this would require separate bidder instances per exchange (and it's more likely to manage that by writing gcloud scripts that create these instances, since the UI doesn't support well a mix of instances with different bidder code deployed in each).

Other conveniences

The exchange connector above is simple, but complete. A real-world exchange will have a much more complex native bidding protocol, and maybe other complications at the HTTP request/response level. But besides supporting these essentials, a really good exchange connector could offer a host of other "conveniences", like:

  1. Decryption of the winning price and any other encrypted fields
  2. Easy access to metadata (e.g., exchange-specific creative attributes)
  3. Decoding or lookup of complex fields (e.g., proprietary geotargeting codes)
  4. Support for exchange-specific macros
  5. Response validation
  6. Reusable interceptors for common steps of request processing
  7. Exchange integration for pixel matching, campaign information, etc.

Features like the above would be hard to support with open standards or even Open Bidder-specific but exchange-neutral APIs, so they're better implemented as exchange-specific APIs. Such features are not essential, but they make bidders much easier to write, as demonstrated by the built-in DoubleClick Ad Exchange connector; look at its source code as a bigger example of how to implement a full-featured exchange connector.

OpenRTB Exchanges

Open Bidder includes full support for OpenRTB, including facilities for implementing a connector for any exchange that uses an OpenRTB-based protocol. We say "OpenRTB-based" because you won't likely find any exchange that supports the pure OpenRTB protocol; extensions are usually required, so the connector has to provide support for exchange-specific OpenRTB extensions.

In the open-bidder-samples module, you will find all code for the MyExchange sample above, and also a MyOpenRtbExchange that's very similar but differs in the supported wire protocol. We're not going to repeat the whole tutorial, just point out the differences. First, here's how you deserialize the request's OpenRTB/JSON message:

BidRequest request = BidRequest.newBuilder()
    .setExchange(MyOpenRtbExchange.INSTANCE)
    .setHttpRequest(ctx.httpRequest())
    .setRequest(jsonReader.readBidRequest(ctx.httpRequest().content()))
    .build();

...and here's how you serialize the bid response to OpenRTB/JSON:

ctx.httpResponse().printContent(jsonWriter.writeBidResponse(response.openRtb().build()));

Looks too easy, right? Just use a pair of jsonReader/jsonWriter objects, provided by the framework. But remember that your exchange will likely have some extensions; we need to know how to handle that. So, MyOpenRtbExchange supports a revolutionary new feature, Circular Ads! (This could work well in smart watches...) Here's an example of a bid request JSON with this extension:

{ "id": "req1",
  "imp": [{
    "id": "imp1",
    "banner": { "ext": { "radius": 120 } }
  }]
}

The first step is defining the model for our extensions, which you do by writing a Protocol Buffer descriptor:

syntax = "proto2";
package com.google.openbidder.sample.openrtbexchange.model;
option java_outer_classname = "MyExt";
import "openrtb.proto";

extend com.google.openrtb.BidRequest.Imp.Banner {
  optional int32 radius = 200;
}

Our extension is defined as a single integer field; you can also create message types to group multiple fields in a single extension (see unit tests in the openrtb-core library for examples of this). Notice that the extend declaration doesn't mention any intermediary ext node because that will exist only in the JSON but not in the protobuf-based model, as you will see. Just like regular fields, each extension must be assigned a unique numeric ID; check the openrtb.proto file to see which objects support these extensions, and which numeric IDs can be used.

The next task is teaching Open Bidder's OpenRTB JSON serializer to handle these extensions we have invented. The serializer itself is extensible; exchange connectors can provide code that will do the serialization or desserialization of new properties. We'll show only the code that configures all that in the Guice module:

@Provides @Singleton
public OpenRtbJsonFactory provideOpenRtbJsonFactory(JsonFactory jsonFactory) {
  return registerMyExt(BidModule.registerObExt(
      OpenRtbJsonFactory.create().setJsonFactory(jsonFactory)));
}

public static OpenRtbJsonFactory registerMyExt(OpenRtbJsonFactory factory) {
  return factory
      .register(new MyOpenRtbExchangeExtBannerReader(),
          BidRequest.Imp.Banner.Builder.class)
      .register(new MyOpenRtbExchangeExtBannerWriter(),
          Integer.class, BidRequest.Imp.Banner.class);
}

You need to create objects that implement the interfaces OpenRtbJsonExtReader and OpenRtbJsonExtWriter, register them in an OpenRtbJsonFactory, which can be bound for dependency injection so the receiver can easily create the worker OpenRtbJsonReader and OpenRtbJsonWriter. Check the open-bidder-samples project for full sources. Notice that extensions can be arbitrarily complex: they could have child message types, arrays, anything allowed in a JSON subtree.

Bid interceptors, unit tests, or other code that manipulates the RTB model, can handle these extensions in the same way you've already seen:

someImp.setBanner(Banner.newBuilder()
    // set regular properties: id, w, h, etc.
    .setExtension(MyExt.radius, 120));

Enviar comentarios sobre…