reworkd
Enhancement: Option to change OpenAI API Endpoint
There are OpenAI API compatible endpoints for other models, such as Basaran: https://github.com/hyperonym/basaran
Adding an endpoint variable to the .env would allow users to use any LLM with AgentGPT, not just OpenAI's endpoint.
This is hugely useful for a number of reasons. But the biggest one is that local models don't rack up an API bill.
Another use case for this is proxy endpoints which log the requests/responses. Lots of people use OpenAI's models with endpoint proxies (myself included).
Hi @MarkSchmidty that's a good idea.
Unfortunately, OpenAI's official node.js library does not support changing the API URL endpoint.
How would you suggest approaching this?
My humble suggestions:
- Expand the official client library to support custom URLs or,
- Find a 3rd party node.js client library that already supports custom URLs
OpenAI's node.js library has a basePath option.
Here are some issues from the OpenAI node.js repo discussing using this option with proxies and alternative endpoints:
https://github.com/openai/openai-node/issues/85#issuecomment-1477432919
https://github.com/openai/openai-node/issues/53#issuecomment-1426254063
Well pointed out thanks. In that case, it should be rather simple to add support for this.
I will add an option for specifying this in the env file.
I have created a pull request for this https://github.com/reworkd/AgentGPT/pull/84
@hahayusuf As someone pointed out in the PR, some countries do not have access to the OpenAI API directly and must use proxies. An option to set this variable in the UI Settings would open up your hosted version to many people who cannot use it in its current form.
Something like this:
Â
Added support for custom base url via an ENV for local development: https://github.com/reworkd/AgentGPT/pull/543
