Patsakula Nikita

@LaurentMazare, hello! Can i ask for review?

> Looks reasonable, could you rebase this branch? Hopefully this should get rid of the clippy error in the CI. Hello @LaurentMazare! I rebased my branch on new `main`, added...

Hello @guillaume-be! I've made a few assumptions that need a sanity check: 1. According to [this](https://huggingface.co/facebook/nllb-200-distilled-600M/blob/main/config.json#L5) line I assumed that encoder and decoder of M2M should be NLLB compatible. 2....

> Thank you @npatsakula for this PR. > > I would be in favor of actually moving from `&str` to `Path` instead of `AsRef` for 2 reasons: > > 1....

Hello @guillaume-be! I reverted to `AsRef` (doc-comments too).

Hello @guillaume-be! I bring some news about adding NLLB support: 1. I added support of special tokens configuration in https://github.com/guillaume-be/rust-tokenizers/pull/76/commits/c13ff10799389e0a6d98d887f36a3cf75880eeb8: it allows to work with config files such [this](https://huggingface.co/facebook/nllb-200-distilled-600M/blob/main/special_tokens_map.json). I...

Hello @tmcdonell! Sorry for delay, I've made a fix using TH.

Now it compiles but tests are broken with some internal error and I don't know what to do. ```text nested data-parallelism mvm: FAIL (expected) (0.05s) src/Data/Array/Accelerate/Test/NoFib/Sharing.hs:70: *** Internal error in...

Hello @dpvanbalen! Yes, you're right: I checked on latest master and catch the same error.

@tmcdonell, unfortunately, it didn't help:( Can we merge it like that or do I need to investigate further?