ferret

ferret copied to clipboard

Add prediction in batches for LOO&faithfulness evaluations (model wrapper: _get_class_predicted_probabilities_texts)

elianap

emanuele-moscato

Dependency issue

3

* ferret version: 0.4.1 * Python version: 3.9 * Operating System: MacOS ### Description I'm following the getting started of ferret framework on macOS and counter several technical issues (due...

nguyen-duchau

Any plans to include seq2seq models support??

2

Thanks for rolling out an excellent model explainer package. Is there any plan for future to support seq2seq models?

techthiyanes

When we visualize faithfulness and plausibility metrics we use a color scheme where, generally, the darker the cell, the better. However, color ranges are unclear and seem to be different...

g8a9

As far as I know, [LIME library](https://github.com/marcotcr/lime) generates rationales for the vocabulary of the sentence i.e. relevance values are predicted for each unique subtoken. For example: ``` exp = explainer.explain_instance("Hello...

ritwikmishra

Opposing results when comparing to transformers interpret

5

* ferret version: 0.4.0 * Python version: 3.9.2 * Operating System: Linux Debian ### Description When comparing your feature attribution scores of the explanation provided by Integrated Gradients (plain) with...

racia

Part of our API accepts human rationale annotations to evaluate explanations on plausibility metrics. It is reasonable to assume that human workers will annotate text with such rationales using open-source...

g8a9

Our current visualization is based on styling pandas tables. This is fine with short sequences where we need to inspect a few items: table visualization does not fit well with...

g8a9

Inconsistent predictions (confidence values) with multiple runs

4

* ferret version: 0.4.1 * Python version: 3.10.9 * Operating System: Ubuntu 20.04.5 LTS ### Description Describe what you were trying to get done. I am loading ferret's explainer with...

kgarg8