tokenizers icon indicating copy to clipboard operation
tokenizers copied to clipboard

`Encoding` object stub doesn't include `__len__`

Open thearchitector opened this issue 1 year ago • 4 comments

It looks like the Encoding object returned by Tokenizer.encode has the __len__ dunder method, and running len(encoding) works, but for some reason it's corresponding generated .pyi file does not. This causes type checking errors in Pyright/Pylance, even when running in non-strict mode (since this library isn't typed).

thearchitector avatar Jun 18 '24 15:06 thearchitector

Ah I see. If you want to open a PR for a fix feel free to do so! Otherwise I'll do it when I have a bit of time!

ArthurZucker avatar Jun 21 '24 08:06 ArthurZucker

I can make a PR if you can point in me in the right direction. Otherwise, I have no problem waiting.

thearchitector avatar Jun 21 '24 14:06 thearchitector

not stale

thearchitector avatar Jul 22 '24 02:07 thearchitector

not stale

thearchitector avatar Aug 22 '24 14:08 thearchitector