This document provides comprehensive answers to frequently asked questions
regarding Meridian GeoX.

## General Meridian GeoX concepts

#### What's the difference between GeoX and user-based
lift measurement solutions?

Balancing measurement precision and user privacy, GeoX and user-based
measurement solution methods rely on fundamentally different approaches:

- GeoX groups audiences by geographic regions (geos) and uses aggregate data, completely ignoring user-level attribution.
- User-based lift tracks individual cookies or ad impressions to attribute conversions.

#### Why should I use GeoX?

Using Meridian GeoX provides the following benefits:

- **Cross-publisher incrementality measurement**: By setting up multi-cell designs, GeoX allows the measurement of cross-publisher incrementality, such as Google versus non-Google media, which is usually not supported by publisher-specific user-based conversion lift.
- **Offline and omni-channel measurement**: GeoX is the primary methodology for measuring offline and omni-channel impact such as in-store sales and CRM-tracked conversions. Because it uses aggregate regional sales data, it avoids the difficulty of linking online user-level ad impressions to offline store transactions.
- **Privacy-durable approach**: GeoX relies on aggregate, unattributed conversions in a specific region rather than tracking individuals. Because of this, GeoX is immune to measurement gaps caused by modern privacy challenges such as cookie deprecation and iOS restrictions.

> [!NOTE]
> **Note**: Geo-based experiments have higher inherent noise because they include all attributed and unattributed conversions in the region. Consequently, they often require a higher financial investment and longer duration to reach statistical power compared to user-based lift.

## Meridian GeoX versus legacy Google GeoX libraries

#### What's the difference between Meridian GeoX open
source library and other Google GeoX open source software, such as
time-based regression matched markets (TBRMM)?

The Meridian GeoX library is a comprehensive, unified upgrade that offers
the following capabilities and features:

- **One-stop shop**: Acts as a single library for both study design (pre-test planning) and analysis (post-test inference), covering multiple methodologies. GeoX supports the time-based regression (TBR) methodology.
- **Direct design comparison** : Includes a `compare_designs` feature to directly compare different study designs side-by-side to see which best fits budget and minimum detectable effect (MDE) requirements.
- **Modern statistical approach**: Uses JAX-optimized computations for fast candidate generation and design-aware placebo inference, improving power and precision in confidence interval estimation.
- **Multi-cell and flexibility**: Natively supports multi-cell designs and allows flexible constraints like excluding specific geos or forcing geos to control.

#### Why does Meridian GeoX recommend a significantly
higher budget and larger MDE compared to TBRMM?

The differences in recommendations stem from different design
philosophies and validation techniques of TBRMM and Meridian GeoX:

- TBRMM estimates design power based on the pre-test in-sample fit. The same historical data is used to fit the model and estimate its variance. In ideal situations, the estimated variance using this procedure should converge to its true variance. However, in reality, the pre-test variance could often be underestimated compared to what is experienced during the actual test. When the actual post-test variance turns out to be higher, the study can end up underpowered. This historical optimization can therefore lead to optimistic or lower budget and MDE recommendations.
- Meridian GeoX addresses this by incorporating out-of-sample validation or honest data-splitting, and placebo testing during the design phase. By evaluating the design's stability on validation periods that weren't used for model fitting, it captures a more realistic estimate of the post-test variance, leading to more conservative budget and MDE recommendations.

Each method is valid under its own technical assumptions, but Meridian
GeoX is designed to build in a buffer against more realistic post-design
volatility, ensuring that experiments have high probability of delivering
conclusive results.

## Data requirements and ingestion

#### Can I use net revenue which can include negative
values due to refunds as the conversion metric?

No. While the schema validation doesn't explicitly block negative
conversions and only checks that the total conversions is positive, negative
conversion values can disrupt the candidate generation (greedy assignment)
and statistical modeling. It's recommended to use non-negative, absolute
values like gross revenue or gross conversion counts. If refunds are
significant, run the analysis on gross sales and apply a historical
net-to-gross ratio post-test.

#### Is campaign spend data always mandatory during the
design phase, and how do I source it?

The campaign spend data requirement depends on your experiment type:

- **Go-dark and heavy-up experiments**: Daily geo-level spend data is mandatory to calculate baseline spending levels and project test budgets. The validation will fail if the spend column is missing for these experiment types.
- **Holdback experiments**: Optional for net-new campaign launches during the design phase. However, you should provide a cost per incremental conversion (CpIC) reference to estimate budget requirements.

The spend data you extract must be daily and aggregated at the same
geographic level as your conversions, such as DMA, state, or postal code.

- **Google Ads** : Extract daily geo-spend using the Google Ads API through reports like `campaign_location_target_report`.
- **Other publishers**: Use publisher-specific APIs or reporting UI exports to retrieve daily spend by geographic target.

Ensure that geographic identifiers match exactly between your conversions
and spend datasets. If you're using postal or ZIP codes, make sure preceding
zeros are preserved as strings during extraction and export to prevent
validation matching failures. For example, your postal or ZIP codes should
be "02138" instead of "2138".

For more information, see
[Prepare your pretest
data](https://developers.google.com/meridian/geox/prepare-your-pretest-data).

#### Our daily conversion data is highly volatile. Can we
upload weekly data instead to smooth out the noise?

No. GeoX strictly requires daily time-series data. Weekly data isn't
supported and will trigger validation errors. To handle daily volatility,
consider:

- Extending the test duration, such as changing from four weeks to six or eight weeks to accumulate more data points potentially lowering MDE.
- Using a shallower conversion KPI, such as add-to-carts or sign-ups, that has higher volume and fewer zero-count days.
- Increasing the target treatment group size or maximum conversion percent.

#### Does the Meridian Geox library allow the ability to
include more than one KPI?

No. The Meridian GeoX library only supports a single conversion KPI per
run, which is the `conversions` column in the data schema. This
means that you can't simultaneously optimize a design for multiple distinct
metrics, such as:

- Segmented conversions, like tracking new and returning buyers as separate target KPIs in the same run.
- Different conversion actions, like tracking store visits and sales simultaneously, or sign-ups and purchases.

A separate design is required for each KPI, as each has its own
historical volume, trend, and volatility. If you run the design engine
separately for each KPI, they'll likely recommend different geo-splits and
budgets.

> [!TIP]
> **Tip:** If you need to measure two KPIs in one test, you should design the experiment using a primary KPI to ensure the test is sufficiently powered. This is typically the conversion KPI with lower volume or higher volatility. You can then analyze the secondary KPI post-test by running the `analyze` function on the secondary KPI's data. Note that the statistical power for the secondary KPI might be lower than planned.

## Study design and constraints

#### Can we force a specific market to treatment or
control group, and how does this impact study design?

- **Force to control** : You can specify geos to be forced into the control group using `included_control_geos` in `Constraints`.
- **Exclude geos** : You can exclude geos entirely from the experiment in scenarios such as protecting key revenue markets or avoiding media disruptions using `excluded_geos` in `Constraints`.
- **Force to treatment**: Not supported in the core library.

If there are no business specific reasons or outlier geos, forcing geos
to control or excluding them restricts the randomization pool and may
degrade model fit. This might result in a larger MDE to achieve statistical
significance.

#### Are my geo units too few for GeoX to be effective?

You should include a minimum of 10 geos for a single-cell design.

- **Use random sampling for 10-20 geos**: In designs with fewer than 20 geo units, stratified sampling is less effective because clustering into strata is constrained. If stratified sampling doesn't yield a satisfactory design for you, you should design with a simpler random assignment.
- **Use stratified sampling for \>20 geos**: When your design has few geos, statistical power is likely to be low, leading to high MDEs or requiring very large budgets. For optimal results, a larger number of geos---for example, ranging from 50 to more than 100---is recommended to allow effective stratification and robust matching.

#### What if I have a flexible budget and want to input a
target iCPA or iROAS and get a required budget?

This depends on your experiment type:

- **Holdback experiments** : You input the target CpIC using `cost_per_incremental_conversion` in `DesignConfig`. If conversions represent revenue, CpIC is equivalent to `1/iROAS`. The library then calculates the required budget to detect the MDE with the preferred power.
- **Go-dark and heavy-up experiments** : You don't input the target CpIC. However, the budget is determined by the historical spend in the selected treatment geos. You can apply a `budget_pct` constraint---for example, -100% for go-dark or a percentage increase for heavy-up experiments---and the library projects the design implied CpIC that you can detect with that budget.

#### Clarification on budget and MDE

The statistical MDE or the minimum percentage change in conversions that
the experiment can detect is determined solely by the historical volatility
and matching of the geos. Changing the budget doesn't alter this statistical
limit.

While the MDE remains constant, the efficiency targets you can
successfully measure do change with the budget:

- **Higher budget**: Lets you detect a higher, less efficient CpIC, or a lower, more conservative iROAS. This increases the likelihood of a conclusive test for campaigns with moderate performance.
- **Lower budget**: Restricts you to measuring campaigns with a very low CpIC or very high iROAS. If the actual campaign performance is moderate, the test may be inconclusive.

#### How should we decide if a design is viable other than
required budget and MDE?

Evaluate the viability using the following:

- **R-squared (\\(R\^2\\))**: \\(R\^2\\) ≥ 0.8 is recommended. A low \\(R\^2\\) such as less than 0.5 indicates the control geos don't predict the treatment geos well, making the post-test analysis unreliable.
- **AA test p-value**: The library filters designs using placebo simulations. Ensure the selected design passes the A/A test (p-value ≥\\(\\alpha\\)) to minimize false positive risk.
- **Feasibility of MDE**: Ensure the projected MDE is business-viable, meaning you believe the campaign can actually achieve a lift larger than the MDE.

#### R-squared adjustment

The minimum required \\(R\^2\\) is controlled by the `min_r2`
parameter in `geox.DesignConfig`. By default, the minimum
required \\(R\^2\\) value is 0.8. If the design search returns no viable
candidates at 0.8, you have the flexibility to relax the threshold, such as
adjusting it to 0.75 or 0.70:

```python
      design_config = geox.DesignConfig(
        experiment_duration=datetime.timedelta(days=28),
        experiment_types=geox.ExperimentType.HEAVY_UP,
        methodology=geox.Methodology.TBR,
        min_r2=0.7,  # Adjust threshold if 0.8 yields no candidate designs
        )
    
```

> [!WARNING]
> **Warning:** While lowering `min_r2` allows more candidate designs to pass the filter, setting it too low such as less than 0.5 isn't recommended, as poor pre-test fit compromises model reliability and inflates inference error.

## Experiment implementation and post-test analysis

#### If I'm running GeoX for the first time, what are the
general guidelines to implement my experiment?

For first-time GeoX users, refer to the following general guidance:

- **Select your setup pathway** :
  - **Platform UI (manual setup)**: Best for standard studies.
  - **Platform editor (bulk setup)**: Best for complex or multi-campaign studies. This requires generating and importing an editor backup file.
  - **Platform API (automated)**: Best for enterprise-level programmatic management. Note that ZIP code targeting isn't supported through the API.
- **Determine campaign setup strategy**:
  - **Modify directly (keep same campaign)**: Use when budgets are uncapped. Restricting geos directly keeps historical machine learning intact.
  - **Duplicate (copy-and-paste campaign)**: Mandatory when campaigns have capped daily budgets. Simply restricting geos on a capped campaign will cause the algorithm to spend the saved budget in the remaining control geos, contaminating the control group. Duplication isolates the budgets.
- **Targeting configuration**: Apply the geo splits strictly using presence targeting to ensure ads are only served in treatment geos and excluded from control geos.
- **Plan for a cooldown period**: After the test ends, maintain a 1-2 week cooldown period before returning to business-as-usual (BAU) to capture lagged conversions.


For detailed instructions on how to execute your experiment in Google,
refer to the [step-by-step
campaign implementation guide](https://support.google.com/google-ads/answer/18073432) in the Google Ads Help Center.

#### Can we keep user-level tests active on the same
campaigns during a GeoX experiment?

No. All user-level measurement studies such as conversion lift and brand
lift should be deactivated on the campaigns under evaluation during the
experiment. User-level studies create hidden randomized holdouts within your
geos, which contaminates the treatment and control split, dilutes
statistical power, and biases the post-test analysis.

#### Does a multi-cell design allow me to test the
statistical significance of the difference between two active treatment
arms?

No. Results for each treatment cell are evaluated independently against
the shared control. The library doesn't provide a direct statistical metric
to test if the difference between two active treatment arms is significant.