This document details how the Meridian GeoX framework utilizes counterfactual
modeling to accurately measure the true incrementality of your marketing
interventions.

## Average treatment effect on treated

Counterfactual or potential outcome modeling is a standard approach in
causal inference. It's essential to measure the average treatment effect on
treated (ATT) or the true incremental impact of your marketing interventions.

Mathematically, the expected incremental impact on conversion outcome \\( Y \\)
following a marketing intervention is defined as:
$$ATT = E\[Y (1) - Y (0)\|D = 1\]$$

In the equation:

- \\( D = 1 \\) denotes the treated geos based on study geo assignment rule.
- \\( Y (1) \\) is the potential outcome of these geos with treatment (factual).
- \\( Y (0) \\) is the potential outcome without treatment (counterfactual).

During the test period of GeoX, only \\( Y (1) \\) can be observed.
\\( Y (0) \\) is unobserved and is predicted by GeoX's modeling approaches.

## Machine learning problem under business-as-usual

The key assumption in GeoX is that without a newly added marketing
intervention, all geos should all be under business-as-usual (BAU) even though
their geo level conversion data may experience some natural fluctuations such as
seasonality.

Using the BAU condition, GeoX builds a machine learning model that
predicts the conversion of treated geos from the control pool. This model can
further help predict the counterfactual metric \\( Y (0) \\) using the control
geos which remain at BAU even after marketing intervention has been placed on
treated geos. Thus, the loss function in the learning problem is calculated as:
$$\\sum_{t \\in \\text{pretest}} \\\|f(Y_{t, T}(0)) - g(Y_{t, C}(0))\\\|_2{ }\^2$$

In the equation:

- \\( Y_{t,T} \\) and \\( Y_{t,C} \\) are vectors of conversion at time \\( t \\) of all treated and all control geos, respectively. The loss function sums up the L2 norm of the residuals by comparing the difference between treated units and control units with only pretest data.
- Functional \\( f \\) is usually defined manually, taking an average conversion at the same time \\( t \\) across all treated geos.
- Functional \\( g \\) can be learned from pretest data by minimizing the total loss function. Different analysis methodologies come up with different values of \\( g \\). In [time-based regression (TBR)](https://developers.google.com/meridian/geox/time-based-regression.md.txt), \\( g \\) is a linear transformation applied on the average conversion across control geos.

Like typical machine learning problems, sometimes it's quite meaningful to add
the L1 or L2 regularization term in the loss function.

The core of the analysis module is that GeoX predicts what your conversion
counts or revenue would've been for the treatment group if you hadn't made any
marketing interventions, and then estimates the incremental effect accordingly.
Examples of marketing interventions include new campaign activation, shut off
media, and increase in spend.

> [!NOTE]
> **Note:** Counterfactual modeling is supported by [time-based regression](https://developers.google.com/meridian/geox/time-based-regression.md.txt).

## Assumptions and implications of violations

Most geo-experimentation methodologies used to establish counterfactual
conversions---including Meridian GeoX---are formally grounded in sharing a common
latent factor structure
([Bai, 2009](https://onlinelibrary.wiley.com/doi/abs/10.3982/ECTA6135);
[Abadie, Diamond, and
Hainmueller, 2010](https://doi.org/10.1198/jasa.2009.ap08746)).

In plain terms, this framework assumes that the daily conversion volume in a
given geo is shaped by broader, time-varying market forces, such as national
media campaigns, seasonality, or macroeconomic trends (referred to as the
**common trend assumption**). While these forces affect the entire market,
individual markets can react with different levels of sensitivity. For example,
the winter season might cause a dramatic drop in retail in New York while having
almost no impact in Florida.

Under this premise, Meridian GeoX designs experiments by identifying treatment
and control groups of geos whose combined sensitivities align and that have high
predictive power for each other, thereby maximizing test precision, specifically
by lowering the
[minimum detectable effect (MDE)](https://developers.google.com/meridian/geox/glossary.md.txt#minimum_detectable_effect_mde)
at the design stage.

If the real data violates this assumption of a shared, stable latent factor
structure among geos in the study, the analysis results may become underpowered
(wide confidence intervals and an inconclusive result), miscalibrated
(confidence intervals that are too narrow, so the false positive rate
significantly exceeds your target significance level,
[alpha](https://developers.google.com/meridian/geox/glossary.md.txt#alpha)), or biased (the counterfactual
systematically over- or under-predicts).

- **Shared:** Treatment and control geos should be driven by the same underlying market forces so that the control group reliably carries information about the treatment group's counterfactual behavior. If geos don't share these common forces, candidate designs fit poorly, failing to reach the required pretest out-of-sample R-squared threshold (`min_r2` in [`DesignConfig`](https://developers.google.com/meridian/geox/api-reference.md.txt#designconfig), default `0.8`) or failing the pretest A/A test (detecting false incremental lift on historical data when no marketing intervention occurred). GeoX uses these pretest checks to screen out many poorly fitting designs during setup.
- **Stable:** What is estimated in the pretest period should continue to hold during the test period, because the counterfactual model is fit on pretest data and extrapolated into the test window.

### Common violations

Common violations include:

- **Spillover Effects (Media Bleed):** Occurs when marketing targeted at a treatment geo spills into control geos, contaminating the baseline and underestimating incremental impact.
- **Local Shocks:** Sudden, localized events (such as regional weather or local promotions) that affect only a few geos, breaking the shared trend. The timing determines the consequence:
  - During the pretest period: The shock degrades model fit and surfaces through the design-stage checks as an R-squared check failure or A/A test failure. These errors indicate the pretest data doesn't support a reliable counterfactual; the remedy is a cleaner pretest window.
  - During the test period: A disruption during the active test window alters market trends and distorts counterfactual predictions. Avoid scheduling experiments during windows with known regional disruptions, such as local promotions or major market changes.
- **Lack of Support (for example, Mega Geos):** Occurs when a treated geo is so unique or large in scale (such as New York City or Tokyo) that its behavior can't be replicated by any combination of control geos.
- **Geo-Specific Trends (Random Walks):** Some geos drift along their own path over time, on top of the market forces shared by all geos. Examples include a gradual local expansion or a slow local competitor entry. Statistically, each such geo follows its own random walk. These geo-specific movements build up over time instead of averaging out, so the relationship between treatment and control geos doesn't stay stable, violating the common trend assumption. Even if a candidate design passes pretest R-squared and A/A checks by chance, the accumulating drift during the test period causes counterfactual predictions to diverge, leading to overly narrow confidence intervals and an inflated false positive rate.

In practice, while violations of these assumptions aren't ideal, mild
departures don't necessarily invalidate the results. For example, a small
amount of spillover from commuters traveling across geo boundaries is often
unavoidable. When the shared market signal still dominates and the design passes
pretest R-squared and A/A checks, the impact is typically minor and the
estimates remain practically useful.

By contrast, in the extreme case where geos share no common market forces and
each geo moves along its own independent path (for example, each geo follows its
own random walk, as in the right panel of the following figure), the common
trend assumption is entirely violated and counterfactual modeling has no valid
basis.

![Two line charts of simulated geo-level conversions. Left: geos follow a common trend with different sensitivities. Right: each geo follows its own independent random walk.](https://developers.google.com/static/meridian/images/meridian-geox-common-trend-assumption.png)

**Left:** Geos share a common trend and react to it with different
sensitivities. The common trend assumption holds. **Right:** Each geo follows
its own independent random walk. The common trend assumption is violated.

### Placebo screening and R-squared thresholds

Meridian GeoX quantifies uncertainty by running the analysis on placebo designs
(see [Robust inference](https://developers.google.com/meridian/geox/robust-inference.md.txt)), simulating
pseudo-experiments on control geos to measure baseline market noise. To protect
confidence interval accuracy, GeoX screens placebo design quality using an
out-of-sample R-squared check (`min_placebo_r2` in
[`AnalysisConfig`](https://developers.google.com/meridian/geox/api-reference.md.txt#analysisconfig), default `0.6`).
When control markets are too volatile or diverse, GeoX raises a `ValueError` for
insufficient valid placebo candidates rather than building intervals on poor
matches.

> [!CAUTION]
> **Caution:** If your study is blocked by the design-stage pretest fit check (`min_r2`, default `0.8`) or the analysis-stage placebo screening check (`min_placebo_r2`, default `0.6`) because R-squared is less than the required threshold, you can lower these thresholds so the design or analysis can proceed (see [R-squared adjustment](https://developers.google.com/meridian/geox/config-your-preferred-design.md.txt#r-squared-adjustment)). However, be aware of the tradeoff: lowering these thresholds accepts poorer-quality matches. If your treatment markets are unique or volatile (such as when treatment geos lack support from control geos), lowering the threshold can produce confidence intervals narrower than expected, underestimating true uncertainty and risking false confidence in your results.