Daniel Deutsch

Accept compressed files as input to `predict` when using a `Predictor`

10

**Is your feature request related to a problem? Please describe.** I typically used compressed datasets (e.g. gzipped) to save disk space. This works fine with AllenNLP during training because I...

Add instance IDs to model outputs

5

It would be really useful if the system outputs had IDs which could be used to match outputs across models. For instance, the CNN/DailyMail outputs are not in the order...

Add a minimum model version

2

The local pipeline hello world code was failing with the error `AttributeError: type object 'edu.illinois.cs.cogcomp.annotation.BasicTextAnnota' has no attribute 'createTextAnnotationFromListofListofTokens'` It turns out it was because the models I had in...

BART included twice

1

Hi, Thanks for this exciting work. Both the dataset and analysis will be very useful! It looks like BART is listed as both extractive (#8) and abstractive (#13). I believe...

https://arxiv.org/abs/1906.03221 https://github.com/ratishsp/mlb-ie https://github.com/ratishsp/mlb-data-scripts

The correlations on the QAGS dataset are near 0, so there is likely some error (either here or with the QAGS data). I do not know what yet.

Adds a wrapper around the QAGS metric. The correlations currently do not reproduce the paper's results and the scores are non-deterministic.

During the correlation calculation, multiple metrics objects are merged into one https://github.com/danieldeutsch/sacrerouge/blob/f1a864cd8b89542466f50315b249e532c3d0ca9a/sacrerouge/commands/correlate.py#L34-L38 If someone calls correlate multiple times in a row, it will mess up the results. This should not...

Add documentation for the stat sig test command: https://github.com/danieldeutsch/sacrerouge/blob/master/sacrerouge/commands/stat_sig_test.py

If a metric is not installed (e.g., a metric like `MoverScore` might not have the necessary pip packages installed, and therefore will not work), the metric-specific command gives a really...