Decomp RSSD – The business fit calibration
During the Meta panel discussion (link in Resources section), I had mentioned about Robyn’s Decomp RSSD metric.
In our earlier posts too, both my co-founder Venkat and I extensively covered the topic of calibration and validation extensively.
Calibration of MMM model can be achieved through goodness of fit.
It is not necessary that statistically well calibrated models make business sense. Therefore one also needs to calibrate the model in terms of making business sense.
During my agency days, once the MMM model was built, we always used to look at the contribution of the marketing channel vis a vis the money spent on it.
This proportion is important to look at because generally it is very rare that channel with low marketing spends has a higher contribution.
This internal ‘business goodness of fit’ helps the modeler to notice any red flags in the model.
Robyn’s Decomp RSSD:
Robyn’s Decomp RSSD accounts for the relationship between spend share and a channel’s coefficient (effect size in a way). If the distance is too far, the result can be deemed unrealistic. For e.g. if one spent $100 on a channel then it should be highly unlikely that it resulted in 100K in sales !!
Validation of a model should only be taken up once the model is well calibrated because validation is an expensive process both monetarily and time wise.
The path to validating the MMM model starts with Goodness of Fit. Make sure your MMM model makes both statistical and business sense.
Resources:
Meta Marketing Science Open Source Measurement Summit – Video Recording https://metaostsummit23.splashthat.com/
