MMM is something that always leads you to reminisce about the learning one had or hadn’t during their statistics course.
15 years ago, when I was a student, I attended one statistics seminar. In it the professor told “Always be testing your assumptions”.
That statement rings true even now, especially in MMM.
In MMM, inference is the name of the game.
In inference, one deeply cares about the preciseness and un-biasedness of the regression coefficients.
So the question arises, what can throw a spanner at your inference?
Heteroscedasticity is one of the culprits.
📌 But what exactly is Heteroscedasticity? Can you simply eye ball it from your data?
One of the assumptions in linear regression is Homoscedasticity which means that once you have fitted the model, the residuals don’t have any pattern.
Residuals offer telltale signs of how consistent your model is. Ideally you would want some kind of stability in your model.
In case of Heteroscedasticity, the residuals by and large have some pattern. Lets take an example of one of the popular pattern (funnel shape).
When you have fan/funnel shape of the residuals, it means the model is getting worse over time since it indicates inflation of error.
📌 Can you eye ball Heteroscedasticity just by looking at data?
No. You can eye ball the residuals and infer heteroscedasticity but not just the raw data.
Heteroskedasticity is an artifact of the model. One would have to build the model first. So pls don’t make the mistake of just looking at raw data and say “my sales is showing Heteroscedasticity”
📌 Why Heteroscedasticity happens?
Heteroscedasticity often happens because of outliers or huge disparity in the range of your independent variables. For e.g. a company spends in the range of 10-15k USD every month on youtube ads. But in few instances, say during BFCM the company decided to really ramp up their spends. Lets say this in the range of 85k-100k. Data like these would lead Heteroscedasticity in the model.
📌 How does Heteroscedasticity affect MMM?
As mentioned earlier, the name of the game in MMM is inference. You want your estimates to be precise and unbiased. However Heteroscedasticity makes your estimates less precise even though it may not bias them.
Heteroscedasticity reduces the trust factor in your MMM. Given that companies make million dollar decisions to spend on certain marketing variables, it becomes imperative that the MMM model you build is trust worthy and accurate.
Statistically, heteroscedasticity would make you infer ‘effect’ even when there is none or relatively small effect because of very low p-values.
📌 What is the solution?
Robust regression could be one of the solution.
To summarize: Statistical rigor is every important. Especially when million dollar decisions are being made based on the model you develop. And as they “Always be testing your assumptions”.
