Hands down Bayesian.
Ask how?
Through priors and posterior distribution.
It is well known fact that one of the Achilles heel of Bayesian technique is the subjective priors.
In a small data (relatively speaking) problem like MMM, one never has enough data where the evidence in data could overwhelm the priors.
This is hence a serious flaw in the technique. But this flaw could also be used in a nefarious way.
I have seen Bayesian MMM analysts confidently specify the location and scale parameters ‘for example – the contribution from youtube should be 7% or that the ROI of a Instagram ad spends should 3.4’.
Remember above that I mentioned there is not enough data to overwhelm the priors. This fact allows MMM analysts to ‘fix’ the contribution % or ROI numbers to their desire. (wink wink ;))
Why do they do this?
They do this because they think these numbers would make the clients happy.
But the repercussion is that, you as a client won’t have the true picture of your marketing effectiveness.
Next lets come to Posterior distribution.
In Bayesian methods, Posterior distribution is the final frontier. All the answer one seeks is in there.
The actions that Bayesian MMM analysts take with respect to posterior distribution is quite befuddling, contradictory and paradoxical.
Bayesians deride frequentists that they rely on point estimates and can’t quantify uncertainty (not true – will explain how in another post).
But when it comes to MMM, Bayesian analysts end up choosing a ‘sort of’ point estimate itself. They choose the expected value (mean) of the posterior distribution as the most likely contribution number of the particular marketing variable.
All the preaching of ‘you don’t quantify uncertainty’ towards Frequentist takes a back seat. The contribution charts of both Frequentist MMM and Bayesian MMM now looks same.
If the mean of posterior distribution is not ‘making sense’ they can choose median or any percentile under the distribution.
You see a Bayesian MMM analyst hence has a lot of ‘room’ (technically the whole probability distribution itself) to select an ‘ideal number’.
What about Frequentist MMM ? Does it not have levers for manipulation?
Frequentist MMM is not impervious to manipulation but it is several magnitude better than Bayesian MMM.
How?
It relies more on Maximum likelihood Estimation.
MLE basically tell you, given the data, what value of parameter is more likely to have generated the data.
Basically the data talks, not the analysts 🙂 I call MLE the truth whisperer.
Because of the above reasons and more (see link in resources), we prefer to use Frequentist methods to fit our MMM model.
Resources:
Want performance guarantees ? choose Frequentist MMM.
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-statistics-marketingattribution-activity-7151460386980945920-H-4U?utm_source=share&utm_medium=member_desktop
Bayesian uncertainty ≠ Frequentist uncertainty
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-statistics-martech-activity-7150376942133776384-DXCG?utm_source=share&utm_medium=member_desktop
Adopting MMM for the first time ? Use Frequentist MMM.
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-marketingattribution-activity-7148928932862472192-ozrW?utm_source=share&utm_medium=member_desktop
Bayesian MMM is not a silver bullet for MMM’s Multicollinearity issue
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-marketingattribution-activity-7106973443807444992-H5Bq?utm_source=share&utm_medium=member_desktop
The Richard McElreath’s Quartet and problem with priors
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-marketingattribution-activity-7107946791983071232-w07F?utm_source=share&utm_medium=member_desktop
Objections to Bayesian statistics
https://www.stat.columbia.edu/~gelman/research/published/badbayesmain.pdf
Expert agreement in prior elicitation and its effects on Bayesian inference
https://link.springer.com/article/10.3758/s13423-022-02074-4
