gumityolcu

Hello, Here are a summary of the contributions: 1. The epsilon values of the batch_norm layers used to be left as they are, when they need to be set to...

**Summary** - LDS metric no longer uses the logits, it uses the probabilities computed by softmax. But this causes underflow issues so we use LogSoftmax to acheive the same scores...

Currently, MixedDataset benchmark's adversarial dataset is not split into train and test samples. Furthermore, benchmark's `download` method uses the base dataset's samples as `eval_dataset`. However, it should be using the...

Lightning module vs. torch model standardization

1

Currently some classes don't expect a lightning module(`ClassDetection` benchmark for example) and there are some discrepancies within classes (LDS benchmark should be expecting both and checking that trainer and model...

We need tutorials that include more explanations, and prefereably not depending on a "all-in-one" model with shortcut, grouped labels, mislabelings etc. We also need to include pages in the docs...

Model device management after training

1

Currently, `SubclassDetection` benchmark will have `self.group_model` in "cpu" even though the original model is supplied in cuda. This raises and error during the `evaluate` call. How to best manage model...

In the randomization test, we need to randomize the model and look at the correlation between randomized and unrandomized explanations. We have a correlation value for each test sample. We...