Sensitivity Analysis for Multiple Treatments

[patriciowoodley]

Hello,

First of all, thank you for such a wonderful library.

I am trying to use Uplift modeling in an A/B testing and I want to validate it.

I have read this notebook about sensitivity analysis but it only takes into account when you have one treatment

https://github.com/uber/causalml/blob/master/examples/sensitivity_example_with_synthetic_data.ipynb

Is it possible to do it for multiple treatments?

I have checked the docs and I am not sure how to pass the p_col parameter in the class Sensitivity because the propensity score is a dictionary in this case and the treatment column is not a binary vector.

[ppstacy]

Hi @patriciowoodley thanks for using CasualML!

Currently, the sensitivity analysis code itself only supports the single treatment use case. But I do think you can run this iteratively for different treatments which means you need to generate the data frame for every treatment and control following the below input. We will have df1 (treatment A + control), df2 (treatment B + control) etc., then use the different sensitivity analysis methods.

Let me know if this makes sense for you. Also if there are also some strong interests from other users, we can plan to extend this to multiple treatments.

[patriciowoodley]

Awesome. That was one alternative but I wanted to be sure that it makes sense.

I will give it a try

Thanks for your quick reply!