frdel
Allow or document the manner in which to use a custom OpenAI endpoint
Hello,
I'm looking at options for internal inference for some models. I know there's configuration for Ollama, yet while this is is great for desktop inference, I am using a remote server instance with more horsepower. Ollama is not a great solution here. I was wondering how to setup an alternative OpenAI endpoint. Perhaps in the model options? If it's not available as an option, it would be nice to have this added, as many inference servers offer this as a standard manner in which to access self-hosted models on more robust inference systems.
Thank you for your work on this project!
+1 - having the ability to specify your own completions endpoint URL/key is TRUE freedom. I'd love to be able to use my LMstudio endpoint for instance, or x.ai's grok3. The flexibility would be great.
Wonderful project!
You can add base_url=http://... and api_key=... to additional parameters field in model settings.
Perhaps a feature request, a tooltip for the additional parameters that are specific to the platform? Or, a bit of a re-org in the docs could really be helpful too, I searched and maybe this is in there somewhere, yet right now things are a mix of legacy and new and it makes it challenging, particularly as I don't fit the Docker Desktop bucket.