Ricardo Rei
Ricardo Rei
Hey Tom, Ok that seems a good idea. I am curious about the languages you tested. We only tested one language that was not covered by XLMR, Inuktitut, for the...
Hi @jlmeunier en-fr has not been used for a while in WMT but a lot of "out-of-english" pairs are and French as a target is also extensively evaluated through the...
@jlmeunier Let me try to give an example. Let's think of annotator A and annotator B for fr-en that are going to annotate the same test set using the same...
Just to add on this: System-level scores are correct, the problem is that we are currently losing the order of the segment-level scores. This should not affect system comparisons but...
I'll investigate this. It seems like a good feature. Do you have support for this in `sacrebleu`? If so I can start by looking into `sacrebleu` implementation
Thanks, Matt! These are nice features indeed! Do you want to submit a PR 😁 ? If not I can still try to allocate some time to do them before...
We were planning the release for the end of November beginning of December
@andmek do you know any python implementation from this method that I can take a look at?
sacrebleu is python only I believe. I'll take a look! thanks!
I refactored the multiGPU inference. This issue is the same as #101. Fix will be merged in next release.