One of the fundamental question that you as a client should be asking an MMM vendor is – “Which technique do you employ to build MMM ? Frequentist or Bayesian”.
Many vendors (predominantly inclined to Bayesian methods) would often try to dissuade you from going in that direction.
Their usual ploy would be to say something along the lines like
▪ Bayesian vs Frequentist – It does not matter. Both are good.
▪ Marketing Effectiveness is not actual science like physics. So cut us some slack.
For the first two points, my rebuttal are the following
📌 “Bayesian vs Frequentist – It does not matter. Both are good.”
The model is the nucleus of the MMM project. Budget optimization, Saturation Curves, Media planning etc. are all artefacts of the MMM.
If you don’t get your nucleus right, none of the downstream outputs are gonna be accurate.
So this debate or introspection needs to be had.
And as you can see from the carousel and the links provided in them, Frequentist MMM turns out to be better.
📌 “Marketing Effectiveness is not actual science like Physics. So cut us some slack.”
I think this argument is a total cop out. Many things can’t be actual science but that should not stop us from endeavoring to make our methods as accurate and scientific as possible.
Most Bayesian MMM vendors say the above because they start off the process with a lot of subjectivity (priors) and then the actual process of fitting the models is such that it feels like a hack.
Asking the clients to lower their expectation on accuracy and robustness because you chose Bayesian methods is simply not done.
Having made the first argument of “It does not matter whether you choose Bayesian or Frequentist”, Bayesian MMM vendors would say “Hey but we use Bayesian MMM”.
If it didn’t matter, why would you advertise that you develop MMM models through Bayesian Methods 😛 ?
Dear clients, So always make it a point to question your MMM vendor on what techniques they are employing to build MMM.
And pls remember, while the attractive Bayesian philosophy of “We update our beliefs based on new evidence” is sold to you, implementation of Bayesian methods to real life problems is a complexity nightmare.
