Quantus
Quantus copied to clipboard
Warn about sample-wise measurements potentially having different lengths
Due to patches being generated non-overlappingly, and depending on attributions, the number of patches per input can differ between samples, leading to different number of measurements between samples.
While this is desired behavior, a warning about that would be great, however, this is not trivial when samples are evaluated instance-wise.
A similar issue happens for region perturbation, but a warning is more easily implemented here, due to the number of evaluations being a hyperparameter (although the same issue as for selectivity can still occur here even without warning)
Writing this issue mainly as a reminder to think about how to deal with this behavior and make it more transparent to the user.