rattle99
rattle99
> Hey I think this is heavily related to the design of Bert Hi @ArthurZucker , true but the word_ids() function is agnostic to the model used I think. Perhaps...
> word_ids() is dependent on the pre_tokenizer, normalizer etc etc I see, that explains why. Thank you!
Thank you! Here is a little code snippet that I worked on for my specific use case when I came across this problem. In case it's helpful. ``` tokenizer =...