MMM: How confident can I be that the results are accurate?

An ROI number on its own tells you very little about how much you should trust it.

The value of the model is directly linked to the quality of its inputs. Imagine two models, one with 100 campaigns to assess, and the other 10. Which would you trust to provide a better prediction? MMM outputs are estimates, not certainties. The key question isn’t just “what’s the ROI?” but “how confident are we that this is close to the truth?”

Two simple checks help answer that.

A confidence interval shows the range within which the true result is likely to sit. For example, an ROI of £1.50 with a wide interval might realistically be £1.00 or £2.00. Narrow ranges suggest strong evidence; wide ranges suggest caution.

A t-test measures how strong a channel’s signal is compared to the background “noise” in sales data. If the uplift attributed to a channel is small relative to normal sales fluctuations, the model can’t be very confident the effect is real. As a rule of thumb, a t-stat above 1.96 indicates a statistically reliable result.

For OOH - where data variation is often limited and spend is sometimes used instead of reach - these checks are especially important. Without reviewing the strength of the evidence, it’s easy to treat an estimate as a solid fact.