LocalAI icon indicating copy to clipboard operation
LocalAI copied to clipboard

Support logprobs

Open vaaale opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. Several use cases require access to the token log probabilities from the completion response. This includes techniques such as FLARE Agents, "SelfRAG" in LlamaIndex, Guidance, as well as, the ability to develop logic to evaluate the confidence level of the generated answer.

Describe the solution you'd like Basically the same as the OpenAI API.

Describe alternatives you've considered

Additional context

vaaale avatar Jul 20 '24 09:07 vaaale