Clarification about DML marginal_effect

[jaydeepchakraborty]

Thank you for such wonderful package. I have categorical Treatment (value = 0, 1, 2) and I used DML.

The effect of the treatments is below.

Then I tried the marginal effect with base treatment = 0.

Now, If I try other treatment value as base treatment such as 1 or 2. It does not change the output. Even whatever value I put as T, the output doesnot change. dml_est.marginal_effect(T=1, X=dml_test_seg) OR dml_est.marginal_effect(T=2, X=dml_test_seg)

Am I missing something? Any help will be appreciated.

Thanks

[kbattocchi]

All of our DML estimators assume that the treatment effect is linear in T (assuming there is no treatment featurizer), so the marginal effect is constant regardless of what values of T you provide, which explains the behavior you're seeing (and you can just call const_marginal_effect instead, which doesn't take T as an argument because we know that the marginal effect is constant in T). So the marginal effect will vary with X, but then we'll just apply whatever coefficients θ(X) we compute to the treatments linearly.

To understand what a marginal effect means for a discrete treatment, we one-hot-encode the discrete treatment (and drop the control treatment's column), so you can interpret the coefficients of marginal_effect as saying what the relative change in the output is to a corresponding change in the treatment from control to T1 in the first column, and from control to T2 in the second column. Note that moving 100% from control to T1 is just what you compute in dml_est_effect_01, so that's why the first column of marginal_effect is identical to it, and likewise for dml_est_effect_02 and the second column. Moving 100% from T1 to T2 is the same as moving 100% from T1 to control and then 100% from control to T2, so dml_est_effect_12 is the same as the second column of marginal_effect minus the first column.

Does that help?

[jaydeepchakraborty]

Thank you very much @kbattocchi ,

• Moving 100% from T1 to T2 is the same as moving 100% from T1 to control (T0) and then 100% from control (T0) to T2.
• It is same as dml_est_effect_12.

So, if I have more than three discrete values (let's say, T0 (value 0), T1 (value 1), T2 (value 2), T3 (value 3)) , We can apply the same rule. dml_est_effect_13 should be same as Moving 100% from T1 to T3 is the same as moving 100% from T1 to control (T0) and then 100% from control (T0) to T3.

Is it correct understanding ?

[kbattocchi]

Yes, exactly. You could either use effect(..., T0=1, T1=3) or equivalently subtract the first column from the third column of const_marginal_effect(...) to get that effect.

[jaydeepchakraborty]

@kbattocchi , thank very much you for the confirmation.

I have one more question and I think it is related. So in treatment, we have three discrete values (0, 1, 2)

Follow are the columns of the DML estimator. print(f"features: {dml_est.cate_feature_names()}, output: {dml_est.cate_output_names()}, treatment: {dml_est.cate_treatment_names()}")

features: ['Dept', 'IsHoliday', 'Temperature', 'Fuel_Price', 'CPI'], output: ['Weekly_Sales'], treatment: ['Type_1', 'Type_2']

now I did below for the shap values.

-- Shap value for the final stage models (const_marginal_effect) dml_shap_vals = dml_est.shap_values(X=dml_test_seg, feature_names=['Dept', 'IsHoliday', 'Temperature', 'Fuel_Price', 'CPI'], treatment_names=['Type_1', 'Type_2'], output_names=['Weekly_Sales'], background_samples=100)

shap.summary_plot(dml_shap_vals['Weekly_Sales']['Type_1'])

shap.summary_plot(dml_shap_vals['Weekly_Sales']['Type_2'])

I have two questions.

1. If I consider the first image, lower the value of CPI higher the output (Weekly_Sales). I am not able to figure out how to fit the treatment (Type : value 1 ~ Type_1) in the figure.

I am assuming, we can say that moving Moving 100% from T0 to T1, the lower the value of CPI higher the output (Weekly_Sales).

Not sure whether the assumption is correct or not.

2. If my above assumption is correct, then how I can plot shap values when I move 100 from T1 to T2?

Thank you again for answering all the questions. I appreciate your help.

[kbattocchi]

I think your interpretation of the first chart is basically correct - it's saying that for the data you have, when CPI is low, the effect of moving from T0 to T1 will be high, and when CPI is high, the effect of moving from T0 to T1 will be low.

For your second question, I believe that the linearity of SHAP values should enable to express the shap values for that treatment direction as a difference in terms of the existing shap values, but our API doesn't currently support doing that.