Amount of data needed

This section can help you build a sense of how much data you need. The guidance about the amount of data needed is rough and directional because the true answer depends on what the data is like.

  • Data size is the number of geos times the number of time points.

  • These time points and geos are not independent. For example, 1,000 data points in a marketing mixed modeling (MMM) setting isn't the same as something like 1,000 coin flips or 1,000 randomly assigned participants in an experiment.

Also see the sections for national models and geo models.

Amount of data for national models

An important confidence check metric for national models is the number of data points per effect that you are trying to measure and understand. For example, if you have 12 media channels, six controls, and eight knots, the total is 26 effects. (For simplicity, ignore things like Adstock and Hill parameters for this example.) If you have two years' worth of weekly data, then you have 104 data points and four data points per effect. This is a low sample-size scenario and you don't have enough data. (Additionally, insufficient variation in the media spend adversely impacts national models.) For more information about knots, see How the knots argument works.

Because it is difficult to get enough data for a national model, you can do the following:

  • Lower the scope of the MMM. You can estimate fewer media channels (either by dropping a channel with low-spend or combining channels), use fewer knots to estimate time effects, and remove any extraneous controls. However, don't remove important confounders.

  • Get much more data. For example, use three years' of weekly data instead of two. Adding more data will reduce the variance in inference, but might make the inference less relevant.

  • Alternatively, consider adding geo granularity to your data and using a geo model instead of lowering the scope or adding more data.

Consider the previous hypothetical example for the national model. You can combine the 12 media channels into three, lower your knots to two. You might also recognize that one of your controls explains the KPI but not the media, which means that it is not a true confounder and you can remove it. If you also use three years' worth of weekly data, you then have 156 data points to estimate 10 effects. This is roughly 15 data points per effect and now you might be able to glean some directional information from the MMM.

Amount of data for geo models

The number of data points per effect that you are trying to measure and understand is still an important confidence-check metric. However, due to the geo hierarchy, that metric is not as clear to interpret. For example, if you have 12 media channels, six controls, 100 knots, and 105 geos, that is roughly $(12 \times 105) + (6 \times 105) + 100 = 1,990$ effects to estimate. (You multiply by 105 for the number of geos because media and controls have geo-level effects.) If you have three years' worth of weekly data, then you have $105 \times (52 \times 3) = 16,380$ data points. This is roughly 8 data points per effect. For simplicity, ignore things like Adstock and Hill parameters in this example.

An important detail that was not considered in this example is that by definition of a geo hierarchy, the geo-level media effects and geo-level control effects are not independent across the geos. Effectively, this means that data is shared when estimating the effect of media channel 1 on geo 1 and the effect of media channel 1 on geo 2. This is similar for controls too. Because data is shared, you effectively have more than eight data points per effect. How much data is shared depends on how similar the effects are across geos. This can be determined by the eta_m and xi_cparameters.

We recommend that if you are having difficulty getting enough data for a geo-level model, then consider combining media channels or dropping a media channel with low spend. Or, you can put a more regularizing prior on hierarchical variance terms eta_m and xi_c, for example, HalfNormal(0.1). The more regularizing hierarchical variance encourages sharing information across geos.

Can I use campaign-level data?

The Meridian model is focused only at channel-level. We generally don't recommend running at the campaign-level because MMM is a macro tool that works well at the channel-level. If you use distinct campaigns that have hard starts and stops, you risk losing the memory of the Adstock. If you are interested in more granular insights, we recommend data-driven multi-touch attribution for your digital channels.