Lee Miller
Results
1
comments of
Lee Miller
It is at least somewhat surprising that this feature is not available. Tiktoken is a 3rd party tokenizer that tries to be compatible with OpenAI's endpoints -- but a tokenizer...