Devon icon indicating copy to clipboard operation
Devon copied to clipboard

Multi Model Config

Open amphetamarina opened this issue 1 year ago • 2 comments

Related Issue: #47

  • Refactor ifs/elifs into separate prepare{model} functions

    • [ ] Discuss if this refactor makes sense
  • Adds 'custom' option on devon configure, giving the user freedom to choose whatever LiteLLM-compatible model they want.

  • Example .devon.config working with mixtral using OpenRouter:

{
  "modelName": "openrouter/mistralai/mixtral-8x22b-instruct",
  "apiKey": "sk-or-v1-<openrouter-api-key>",
  "apiBase": "https://openrouter.ai/api/v1",
  "promptType": "llama3"
}

amphetamarina avatar May 22 '24 16:05 amphetamarina

So, might have going out of the way with the refactorings. Also pulled from main and added changes so not to break ollama support.

I'm doing more comprehensive regression testing here, just wanted to know if the refactor is a GO or not, I can just change back to ifs/elifs if it doesn't make sense to extract them now.

amphetamarina avatar May 23 '24 14:05 amphetamarina

This is great work! Will review tomorrow. Thanks!

akiradev0x avatar May 27 '24 07:05 akiradev0x