The Meridian GeoX framework prioritizes a robust, data-driven approach to
experimental design to minimize variance and advertiser budget requirements.
Moving beyond standard randomized sampling, stratified sampling is proposed to
generate design candidates which ensures that treatment and control groups are
more comparable before an experiment begins. After stratified sampling, the
design search process will filter design candidates and finalize designs that
achieve the greatest statistical power.

## Stratified sampling

Meridian GeoX uses a stratified sampling design to address the variance and high
budget issues inherent in restricted sample sizes.

- **Stratification process**: It partitions geographical units (geos) into "strata," which are homogeneous clusters defined by shared characteristics including response volume, trend and seasonality.
- **KPI featurization**: Before clustering, KPI series are featurized using statistical metrics including the mean, coefficient of variation (CV), slope, autocorrelation (at lag 1), and dynamic time warping (DTW).
- **Clustering and random assignment**: A k-means algorithm can partition geos into strata based on these features. The default number of clusters is four. For each geo, randomized assignment into treatment or control groups is then performed within each cluster. Note that the random assignment also naturally expands to multi-cell study design.

![A chart of the stratified sampling design.](https://developers.google.com/static/meridian/images/meridian-geox-stratified-sampling-chart.png)

## Random sampling

Meridian GeoX supports random sampling, in which geo units are randomly
partitioned to treatment and control groups without following the clustering
structure at all. Random sampling is not generally recommended over the
stratified sampling methodology, however, it becomes necessary in GeoX with a
small number of geos.

## Design search process

Meridian GeoX uses a rigorous design search pipeline to identify optimal
experimental design candidates from a pool of likely designs generated from the
stratified sampling method.

### Evaluation metrics

The following metrics are evaluated under Meridian GeoX to closely monitor the
quality of a design:

- **Out-of-sample fit**: Indicates whether or not the model fits a design, and is evaluated using the out-of-sample R-squared statistic to filter candidate designs.
- **AA robustness**: Validated through A/A simulations where p-values are checked to prevent statistically significant A/A biases.
- **Statistical power**: Measured by the minimum detectable effect (MDE) of the
  relative lift. GeoX MDE lift percentage is independent of the options of
  analysis methodology, and is calculated using the formula:

  $$MDE_{\\,lift} = Min\\:incremental\\:conversion\\:/\\:Baseline\\:control \\:conversion\\\\= SE_{\\,lift} \\times (Z_{1-\\alpha/2}\\:+\\:Z_{1-\\beta})\\:/\\: Baseline\\:control\\:conversion$$

  In the formula, \\( SE_{\\,lift} \\) denotes the standard error of absolute
  lift, while \\( \\alpha \\) and \\( 1-\\beta \\) represent the significance
  level and the target statistical power respectively. By default, a two-sided
  hypothesis testing is considered. \\( Z \\) corresponds to the standard
  normal quantiles.
- **Budget**: Translates the statistical power into the fiscal requirement to
  reach power objectives.

### Search pipeline

The design search process divides the entire pre-test data into initial
training, validation, and final selection periods. The process then follows six
sequential steps to finalize the designs. Refer to the diagram in this document
for the sequential search pipeline steps:

![A chart of the design search workflow.](https://developers.google.com/static/meridian/images/meridian-geox-design-search-workflow.png)