Marketing Mix Modeling (MMM) – Optimize your Marketing ROI

Confidence intervals in a way are barometer of your Marketing Mix Models

A lot of people blame Confidence Intervals and its ‘unintuitive’ nature for switching to Bayesian side of things. But if you are in Marketing Mix modeling domain, frequentist concept of confidence interval makes more sense. Before I elaborate, let me provide a quick recap of what exactly is Confidence Interval. 📌 What is confidence Interval? Lets say we are talking about the popular 95% CI. The definition of it would be – If one ran the same statistical test taking different samples and constructed a confidence interval each time, then in 95% of the cases, the confidence interval so constructed for that sample will contain the true parameter. Now that

Looking to get started on MMM? Know your type of Data.

One key advice I give to statisticians/data scientists looking to get started on Marketing Mix Modeling (MMM) is – Know your type of Data. A big chunk of the projects in the industry involves dealing with tabular data. This is particularly true of Marketing Measurement and Attribution Industry. However, not many have the knowledge of Time Series Data, Cross Sectional Data and Panel Data. All these types of data have a temporal component. It is thus very important to know the subtleties because the type of regression that you would apply will vary slightly depending on the data and business objective at hand. So here is a quick explainer: 1.Time

Why you should not expect high R squared value in your MMM models

Why customers should not expect their Marketing Mix Models (MMM) to have very high R squared value. Somehow over the years, two myth has been propagated : ▪ High R squared value = good ▪ R square is a sign of predictive power of a model I guess we statisticians are partially to blame for propagating these myths. Let’s break these myths: 1) R Squared value is not a measure of predictive power Though some have this misconception, R squared value is more about retrodiction than prediction. R square value is more about ‘goodness of fit’. 2) There is no ideal R squared value Because of the myth that R

MMM pro tip: Use spends instead of Impressions in your model

1. Spends are more actionable and transactional, as you can measure direct impact on sales. Spends data directly reflects the financial investment in marketing (your CFO would be happy 😅). 2. Spends are more accurate than impressions. 3. Cross-Channel Comparisons – Spend data enables easier comparisons between different marketing channels or campaigns in terms of their cost-effectiveness. This is essential for understanding where to allocate resources for maximum impact. 4. Saturation Curves – Saturation Curves with spends give a clear indication of saturation of the medium rather than impressions.

Don’t make million dollar marketing decision just based on correlation

Marketers are sometimes given bad advice that they should not go for advanced methods for marketing impact measurements. Instead they are suggested to adopt simple analysis like correlation. Ill advised suggestions like “All you need is only correlation” will do more harm than good. So I will detail out the advice that I came across in one such post and explain why it is wrong to simply rely only on correlation. The advice goes something like this: Calculate correlation coefficient between CAC and advertising spends. If you observe a correlation coefficient closer to 1, then your ad efforts were a waste. Now why is the above advice flawed? For this

Unpacking the granularity problem in MMM

One of the complaints many have with respect to MMM is that it does not provide granular insights. While there are multiple solutions to this problem (will talk about this in future posts), let me unpack the granularity problem. So what is the Granularity problem? Lets take the example of TV. Many brands spends lot of money on TV ads and the below are typical scenarios: Scenario 1 – different ads/creatives run concurrently. Scenario 2 – the ads/creative run at different cadence across the whole time period. The latter leads to data sparsity and often warrants use of cumulative TV spends/GRPs at the model level. The usage of cumulative spends

Validating MMM models the right way

I often come across posts where people overtly prescribe out of sample error and incrementality testing as the only way to validate the MMM model. They often do so by belittling the importance of goodness of fit. I am sorry to say, but people who say “R squared value and statistical significance measures (e.g. p-values) don’t help in validating the MMM model” simply don’t understand statistics and as an extension don’t understand MMM either. If your vendor is trying to water down statistical significance measures, then you should be really wary of them. MMM models do need to be validated on out of sample data and incrementality testing (with causal

Don’t train-test split your data in Marketing Mix Modeling

In MMM, You don’t need to train/test split your data. Okay, some of you might be shocked since I am going against the conventional wisdom prevalent in ML circles. But let me elaborate. Ideally if your goal is inference, you don’t need to train/test split your data. In case of prediction, train/test split is justified as the model making such predictions is often black box-ish. When the goal is inference, just as it is in case of Marketing Mix Modeling (MMM), train/test split means that your are ‘wasting data’. The 25% or 30% data that could have been utilized for better specification of the model and thereby better understanding of

How Multicollinearity saps the statistical power

If you are familiar with Marketing Mix Modeling (MMM) or just multi linear regression in general, you must have noticed the following effects at some point in time: 1) Signs of variables changing 2) Wide Confidence Intervals 3) Large Standard Errors 4) Inflated R Squared value 5) Overall bad model fit These are tell tale signs of multicollinearity. In the marketing mix modeling space, attribution is everything. Failure to do so is a huge downer. Both Ridhima and I have talked about this issue in many of our posts (links to some in comments). In this post however, I want to talk in depth about the statistical power. Statistical power

How useful is F-test in Marketing Mix Modeling?

As most of you know, one of the interesting application of linear regression is Marketing Mix Modeling (MMM). Even though additional bells and whistles are added in MMM over and above what a traditional linear regression entails, the core of MMM is still linear regression (or its many variants). Given this background, it becomes imperative that you diagnose your MMM model properly. Many misread the diagnostic plots and metrics in Linear Regression. And as a result this extends to wrong interpretation of the MMM model too. So lets delve into what people get wrong about the F test. But before that, let me give you a background of what F

Blogs