The Meridian model specification contains a `holdout` argument (a
[`HoldoutSpec`](https://developers.google.com/meridian/reference/api/meridian/model/spec/HoldoutSpec)) that
can be used to specify a holdout sample. The KPI data of the holdout
observations is ignored during model training (for example, MCMC posterior
sampling), and does not affect the model likelihood or posterior density. Media
data for the holdout observations is still used for model training, because it
affects the adstocked media values for subsequent time periods.

A `HoldoutSpec` takes one of three forms:

- A [`RandomHoldoutSpec`](https://developers.google.com/meridian/reference/api/meridian/model/spec/RandomHoldoutSpec) holds out the same number of randomly chosen time periods in each geo.
- A sequence of [`DateRange`](https://developers.google.com/meridian/reference/api/meridian/model/spec/DateRange) holds out the same dates in every geo.
- A sequence of [`GeoHoldoutSpec`](https://developers.google.com/meridian/reference/api/meridian/model/spec/GeoHoldoutSpec) holds out the given dates in the given geos.

The forms differ in who draws the holdout sample. A `RandomHoldoutSpec` asks
Meridian to make the random draw. With the other two forms, you make your own
draw and express the result as the dates, or the geos and dates, to hold out.
Meridian holds out exactly those observations.

For examples, see [Use a training and test data
split](https://developers.google.com/meridian/docs/user-guide/configure-model#use-training-test-data-split).

The primary use of the holdout sample is for calculating out-of-sample goodness
of fit metrics, such as R-squared. This is useful for comparing different model
specifications, such as prior distribution strengths, provided that each model
being compared uses the same holdout sample. There is no guarantee that the
model with the best out-of-sample model fit is the best model for causal
inference, but generally a better fitting model is preferred. Model
misspecifications that lead to poor model fit can also cause bias in causal
inference.

We recommend using a holdout sample that is fairly balanced across geos and time
periods. In other words, use a holdout sample that has approximately the same
number of holdout observations for each geo and approximately the same number of
holdout observations for each time period. If the holdout sample is imbalanced,
this can result in too few training observations to estimate the geo effect
\\(\\tau_g\\) for certain geos, or the time effect \\(\\mu_t\\) for certain time
periods. By default, Meridian does not specify a holdout sample. You
must specify the holdout sample and ensure that it has a reasonable degree of
balance.

Avoid holding out large, contiguous-in-time, chunks of data, such as at the end
of the MMM time window, to assess forecast error in the KPI. Meridian
isn't designed for forecasting the KPI, especially if there is strong trend and
seasonality in the KPI. Instead, Meridian estimates the causal media
impact and uses the knot-based approach to modeling trend and seasonality. The
knot-based approach needs data near the knot to estimate the knot effectively.
If large, contiguous-in-time, chunks of data are held out, there is no data near
the knots within the held out period. In this case, the knot's posterior
distribution is driven by the prior, which can result in poor forecasting.

Additionally, Meridian can be used to estimate the impact of both
historical and future media because it assumes that model parameters which
determine media impact are consistent over time.