polymath
polymath copied to clipboard
Allow configuring a shorter context to allow a shorter completion.
The completion API sets a limit on the length of (input + output). The longer the context, the shorter the completion must be.
There's a balance between passing a lot of context to summarize and getting a long result.
It should be possible to configure that split in ask.
Originally noticed while thinking about #14
- [ ] Allow setting answer_length in
sample.client. - [ ] Allow setting prompt_length in
sample.client(sometimes you don't want to use the entire prompt; it's more expensive)
Consider tiktoken to measure token length.