Anna Hedström

New data domain: Quantus + NLP = True

3

**Goal**: make Quantus source code compatible with NLP tasks, including the addition of a tutorial to showcase how Quantus can help quantify the goodness of NLP explanation - [ ]...

New metric: implement Relative Stability

1

Implement Relative Stability (belongs to the Robustness category). • **Relative Stability**: leverages model information to evaluate the stability of an explanation with respect to the change in the a) input...

Using e.g., https://readthedocs.org/

Let's make a Metric Cimparison metric, similar to Captum's: https://captum.ai/docs/algorithms_comparison_matrix)https://captum.ai/docs/algorithms_comparison_matrix. Tasks: - [ ] Create a MetricMatrix.md file - [ ] Start adding things like: - Short description e.g., a...

Currently, we rely on a local import but we should make a `!pip install quantus` in all tutorials instead

New tutorial: on how to customise metrics with own user-defined functions

2

Would be great with a tutorial that shows: - how to employ user-defined explanation functions into the tests - how to customise the metric tests like perturb_func, similarity_func etc Maybe...

New tutorial: for how to use Quantus with 1D shapes

1

- Make a tutorial for how to use Quantus with 1D shapes - Reference PR: https://github.com/understandable-machine-intelligence-lab/Quantus/pull/74

Enhancement: out metrics' **kwargs in *args and set default values

1

To make the parametrisation of metrics in Quantus more explicit (in order to mitigate the risks of user typos and other undefined behaviours e.g., when passing a larger kwargs for...

Add mypy to CI/CD

2

Add mypy (http://mypy-lang.org/) to execute static type checking in Quantus CI/CD pipeline

Make more base methods available https://captum.ai/api/!