Inconsistency config and finetune notebooks

[BramVanroy]

Hi there

I wanted to reproduce the results from your paper on the sentiment analysis task. I followed all the steps you list and then ran the notebook for DBRD. I found two unclarities that I hope you could clarify:

• the notebook uses v1, and it is not clear whether the same notebook should be used to reproduce results for v2;
• the Config that is used in the notebook is not the same as the one in the repo: in the repo, the config has gradient_accumulation_steps = 8. However, in the notebook output cells I can see that you originally ran this with gradient_accumulation_steps = 1. I could only reproduce your results when I changed the notebook so that config.gradient_accumulation_steps = 1, while running with 4 GPUs.

Maybe these things can be clarified/made more consistent in the repo?

In the end I was able to reproduce your results. My results are a bit lower than than the ones you report in this repository, but they are within the confidence interval that you described in the paper so thanks for including that CI!