Ofer Mendelevitch
Ofer Mendelevitch
Hey @williamberrios - yes we continuously updating this. Please feel free to reach out to me (ofer at vectara.com) - I would need the name of the model, how to...
1. I am not sure- this is how the results come out. It really depends on what the LLM vendor does during training and what metrics they focus on, which...
Thanks. I went trough the guidelines twice again and updated the PR - thanks for the feedback. Linting locally (npx awesome-lint https://github.com/vectara/awesome-agent-failures) worked okay but it seems to have failed...
thanks @n1ckfg - fixed!
Thanks for the comments - updated these.
I think all fixed now - thanks for the feedabck.
We only evaluate for the leaderboard models that are available via the API. For GPT-5 that is nano, mini and main GPT-5 (minimal/high thinking). Is there another one you were...
I am not sure Finix-S1 is available publicly yet, but should be announced soon.
I would argue that within RAG, most uses would use temp of 0 or close to it. Why would you use a larger value, especially when in large RAG deployments...