MMM: Are there enough instances of individual channel use?
Is the advertising being well managed in the model?
MMM learns from variation and repetition.
If a channel has only appeared a handful of times in the dataset, the model has very little evidence to detect a real pattern. It may fail to detect impact — or wildly overestimate it based on one unusual result.
OOH often suffers here. If it has only run twice in three years, the model simply doesn’t have enough examples to learn its true value.
How many campaigns does the model have for each channel — and is that enough to learn from?
YES!
There are multiple varied campaign examples.
Good - the model has enough evidence to detect consistent patterns. Look for variation in the data across spend, timing, geography. This will proivide enough data to determine the patterns.
NO!
There are only a handful of examples.
With limited data, ROI becomes more guesswork than measurement. This will mean more volatility in the results. Try supplementing the model with additional evidence and adding data such as a longer historical window.
To understand more: