Joannes Vermorel

With the following config, I am able to do a request and get a proper response from the OpenAI served by Azure: ```csharp _api = new OpenAIAPI("..snipped" /*apiKey*/); _api.ApiUrlFormat =...

Add 'TokenLimit' as a property of models?

1

The token limit is an important property when dealing with one model or another. It would be very nice if the `Model` class had a `int TokenLimit` property that gives...

Collapse of performance around ~2M items

1

I have been toying a bit with the Robin Hood dictionary, and the correct implementation seems to suffer from an odd collapse of performance around 2 million items. Here are...

Hello! Thanks for publishing `rust-tokenizers` in the first place! FYI, we have reached our first milestone in porting `rust-tokenizers` to C#/.NET at https://github.com/Lokad/Tokenizers/ This lib is published under an MIT-style...