In MMM, there is often a dilemma on whether to make model better at explanation or prediction.
Some MMM vendors focus on prediction while compromising on explanation.
But is it the correct approach?
No.
At Aryma Labs, we err on the side of caution and focus more on getting the explanation right first.
๐๐ก๐ ๐๐ข๐๐ฌ / ๐๐๐ซ๐ข๐๐ง๐๐ ๐ญ๐ซ๐๐๐๐จ๐๐
Most data scientists understand Bias / Variance tradeoff from the lens of overfitting alone.
But if one came fromย #statisticsย /econometrics, they would
see the bias/variance tradeoff from the lens of data generating process and estimator bias.
๐๐ก๐ ๐๐ข๐๐ฌ:
Bias is generally an attribute of the estimator.
Bias is the difference between estimator’s expected value and the true value of the parameter it is estimating.
An estimator is said to be unbiased if itโs expected value is equal to the parameter that weโre trying to estimate.
๐๐ง๐๐ข๐๐ฌ๐๐๐ง๐๐ฌ๐ฌ ๐ข๐ง ๐๐๐
MMM is all about attribution. There is a true value or ROI of the marketing variable. Through MMM, our job is to hone in on this true ROI.
Our MMM models hence have to be unbiased so that we converge to this true ROI values.
๐๐ซ๐๐๐ข๐๐ญ๐ข๐ฏ๐ ๐๐๐๐ฎ๐ซ๐๐๐ฒ โ ๐๐๐๐ฎ๐ซ๐๐ญ๐ ๐๐ฑ๐ฉ๐ฅ๐๐ข๐ง๐๐๐ข๐ฅ๐ข๐ญ๐ฒ
People often think that MMM models that has low MAPE/ RMSE would be good at explainability of the model too. This is not true. Dr. Galit Shmueli captures this point excellently in herย paper ‘To explain or to predict’.
โช Does good explainability translate to good predictive accuracy?
From our experience, Yes. Models that have less bias automatically capture the data generating process correctly.
If you capture the DGP correctly, you will also have good predictive capability (unless there is a sudden change in the environment or the very characteristics nature of variables included in the model changes).
๐๐จ ๐ก๐จ๐ฐ ๐๐จ๐๐ฌ ๐๐ซ๐ฒ๐ฆ๐ ๐๐๐๐ฌ ๐๐ฎ๐ข๐ฅ๐๐ฌ ๐ฆ๐จ๐๐๐ฅ๐ฌ ๐ฐ๐ข๐ญ๐ก ๐ฅ๐๐ฌ๐ฌ ๐๐ข๐๐ฌ ?
โช We don’t use Bayesian Regression methods.
The priors are the biggest source of bias in your model. As stated in many previous posts, MMM is a small data problem. Your priors will almost always overwhelm the evidence in your data.
โช We use regularization sparingly
MMM’s multicollinearity problem can be solved via regularization. But that also induces bias. We instead focus on methods like residualization to reduce multicollinearity.
โช Control for all variables
One of the leading cause of bias in MMM model is not controlling for all variables in the model. Some vendors might say there is no need to include all variable in the models. But such a model will have a lot of bias. Especially Omitted Variable Bias. This leads to endogeneity and hampers causal inference.
๐๐ง ๐๐ฎ๐ฆ๐ฆ๐๐ซ๐ฒ:
MMM is more about causality than prediction. It is always prudent to develop models with less bias.
Resources:
To explain or predict paper :
https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf
Bayesian MMM vs Frequentist MMM – Key Comparisons
https://www.linkedin.com/posts/venkat-raman-analytics_frequentist-mmm-vs-bayesian-mmm-comparison-activity-7161591828285210624-qyUm?utm_source=share&utm_medium=member_desktop
How we use AIC & KL Divergence in MMM models
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-statistics-datascience-activity-7171760874666254336-Vuv4?utm_source=share&utm_medium=member_desktop
Bayesian MMM’s Stating the obvious problem
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-marketingeffectiveness-activity-7171027667469684736-1j0f?utm_source=share&utm_medium=member_desktop
Which technique provides for great manipulation in MMM – Bayesian or Frequentist?
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-statistics-marketingattribution-activity-7156533790130003968-qVTr?utm_source=share&utm_medium=member_desktop
Adopting MMM for first time ? Use Frequentist MMM.
https://www.linkedin.com/posts/venkat-raman-analytics_marketingmixmodeling-marketingattribution-activity-7148928932862472192-ozrW?utm_source=share&utm_medium=member_desktop
