sinjohr

custom tokenizer and text encoder

1

I want to use custom tokenizer and encoder trained from huggingface tokenizer. After training the huggingface tokenizer, I got a json containing vocas. However, I don't know how to feed...