[REQUEST] Custom OpenAI API URL

[racerxdl]

Is your feature request related to a problem? Please describe.

Currently we're locked in to specific models and we can't even set a custom API url making it unable to use llama-swap, llama-server or even ollama in OpenAI Compatibility mode.

Describe the solution you'd like Add a model listing almost like the Azure/* that allows to write any model name, a custom API URL and a custom token. Internally, it would process as same the OpenAI / Azure client.

Describe alternatives you've considered Azure/* sort of works, but I have to declare the name of my models as the same as azure has which is not great, since, for example, in llama-swap I can declare whatever name I want.

[Aptronymist]

Agreed. Having an OpenAI compatibile option is essential.

[bobbythomas]

Absolutely agree. I'd love an OpenAI API compatible option where we can define any OpenAI compatible provider.

[xhzkp]

Absolutely agree!!!

[fludo]

+1, seems essential