vbougay
Results
2
comments of
vbougay
@zealrant Thanks for your suggestion. I've done the same thing, but I don't feel like tokenizer works correctly anyway. For perfectly normal English string like "one more pseudo generalization and...
It turned out that the file which the script tries to load as a vocabulary in fact is saved SentencePiece model. The change in the lines 159-161 did the trick:...