Dilyara Bareeva

Add random seed for reproducibility

2

Make sure multiple re-runs of each metric deliver the same values each time.

Mypy: fix errors

2

### Implemented changes - fix most errors @annahedstroem Please check the remaining errors. - small edits in CONTRIBUTING.MD - branched from mypy-static-type-checker

quanda link in README.md

1

### Implemented changes - Added a **quanda** link to README.md - Changed pytest.ini configuration to ignore FutureWarning due to failing tests

feat: Unified configuration for benchmark training, unit testing and usage

1

Currently, the process of training and setting up benchmarks is overly complicated—we have to configure them separately in the benchmark classes, tests, and training scripts. I'm proposing a unified approach...

**Describe the bug** When running the tests on **cuda**, I get the error: ``` --- > raise NotImplementedError(error_msg) E NotImplementedError: Automatic batch size search is not supported for multi-GPU setting....

Torch doesn't like it anymore when non-weights are loaded with torch.load. Fix this. - shall we just suppress the warnings and set `weights_only=False?`

As a first step in incorporating text_classification into quanda, we want to adjust `ClassDetection `metric to support text classification.

- this clashes with state loading functions pre-defined in Benchmark base download

- accessing the benchmark models wrapped into a lightning module is awkward right now. Problem A: - nested layer naming. We should create a layer `prefix` string, so that the...

Checkpoints should be saved as separate files in the file storage and accessed separately from the rest of the benchmark dictionary.