How to use AIC to select the best Marketing Mix Model (MMM).
Firstly, let’s look at what is AIC and the most common misunderstanding associated with it.
The Akaike information criterion (AIC) is given by:
AIC = 2k -2ln(L)
where
k is the number of parameters
L is the likelihood
The underlying principle behind usage of AIC is the ‘Information Theory’.
In the AIC equation we have the likelihood. We try to maximize the likelihood.
It turns out that, maximizing the likelihood is equivalent of minimizing the KL Divergence.
But what is KL Divergence?
From an information theory point of view, KL divergence tells us how much information we lost due to our approximating of a probability distribution with respect to the true probability distribution.
When comparing models, we choose the models with lowest AIC because in turn it means that the KL divergence also would be minimum. Low AIC score means little information loss.
Now you know how KL divergence an AIC are related and why we choose models with low AIC score.
Ok, now getting back at choosing the best MMM model through AIC.
One of the misconceptions about AIC is that the AIC helps in choosing the best model out of a given set of models.
However, the key word here is ‘Relative’. AIC helps in choosing the ‘best model’ relative to other models.
For example, if you had 5 MMM models (fitted for same response variable) and all 5 are overfitted badly, then AIC will choose the least overfitted model among all models.
🚩 Note: AIC will not caution that all your MMM models are poorly fitted. In a way AIC is like a supremum of a set.
