Yan Yuanmeng

OpenKiwi always download the tokenizer files for XLMRoberta even if a local path is configured.

2

When I am training the XLM-Roberta based QE system, I pre-downloaded the pre-trained XLM-Roberta model from huggingface's library and modified the field `system.model.encoder.model_name` in `xlmroberta.yaml` from the default `xlm-roberta-base` to...

Fix bug that always downloads tokenizer files.

1

When creating `XLMRobertaTextEncoder` object, the tokenizer name will be rewritten to `xlm-roberta-base` if a local model path is configured, so that the framework will always download the tokenizer files via...