Implemented support for more LLM models
This PR implements support for more LLM models like Claude 2 from Anthropic, PaLM from Google, Llama v2 70B from Meta, and GPT-4 from OpenAI (more models are being added) by routing the request using OpenRouter with change of code and affected compatibility kept to a minimum.
-
Requests to OpenRouter use the same
openaimodule as those sent to OpenAI, no additional dependency is required. The only difference is that it requires aheadersargument for tracking usages from different apps. User can fill out whatever header they like for potentially supporting more routing services. -
The option to use OpenRouter (or other services supported) is opt-in, means that users who doesn't know or don't want to use it won't feel a thing after they upgrade the workflow. And if they do use it, it will only apply to the chat completion part, instruct GPT part and image generation part will still use models from OpenAI.
-
Unsupported parameter will be ignored for models that do not support it, such as
streamformeta-llama/llama-2-70b-chat, in which case a window will show after the full reply is returned. No code modification for each parameter based on each model.
I'll add the missing documentation once if it seems reasonable for you to include this in ChatFred, I'm curious about what you think! @chrislemke
You can get your own key at https://openrouter.ai, $1 of credit is granted with each new account.
Hey @Sponge-bink! Thanks for this. A really cool idea! I will checkout https://openrouter.ai - looks interesting.
I'm now discouraging this PR since the service of OpenRouter has been, to say at least, unstable recently.