Michael Partheil
Michael Partheil
I just ran a short benchmark, on my machine it is 47x faster for encoding than the Rust-based `CLIPTokenizerFast` implementation from `transformers`: ```python from transformers import CLIPTokenizerFast from instant_clip_tokenizer import...
> Have you tried batch tokenization? We mostly care about tokenization performance for single inputs (we use it for inference). Nevertheless, we provide a `tokenize_batch` method which is around 3x...
Sure - opened #180 for further discussions!
This would be a super-nice addition to the crate! Hoping this can be merged soon, thanks for your work!
@asny @paul2t Thank you very much for your efforts and hard work on this! It'd be really great to get this merged, as winit 0.28 depends on the unmaintained `instant`...
I assume the exact same argument also holds for `ExclusiveData`, and hence the trait bounds for that type should also only require `T: Sync`? If others agree #84 changes this.
I'm running into the same issue - I have a fork of `libm` that is used as a git dependency in one of my projects. Now this sometimes fails in...
> [@michael-p](https://github.com/michael-p) I believe the idea is that it works without depending on compiler-builtins at all, through the Rust compiler. So just try and remove the dependency and see if...