LocalAI
LocalAI copied to clipboard
Support logprobs
Is your feature request related to a problem? Please describe. Several use cases require access to the token log probabilities from the completion response. This includes techniques such as FLARE Agents, "SelfRAG" in LlamaIndex, Guidance, as well as, the ability to develop logic to evaluate the confidence level of the generated answer.
Describe the solution you'd like Basically the same as the OpenAI API.
Describe alternatives you've considered
Additional context