TogetherAI compatibilty?

[Jirubizu]

TogetherAI has a openai compatibility api. It allows for simple plug and play with the openai module in python. However, after setting up the .env file as shown below, i get the following error OpenAI API failed with status: 401 error: Incorrect API key provided: sk-vxLAE***************************************tBZb. You can find your API key at https://platform.openai.com/account/api-keys.'}]}, 'status': 'FAILED'}

Now I simply put my .env file in my $HOME directory and I am running verba start from the same directory too. Am I missing something?

.env

OPENAI_API_KEY=TOGETHERAI_KEY
OPENAI_BASE_URL=https://api.together.xyz/v1
OPENAI_MODEL=meta-llama/Llama-3-70b-chat-hf
AZURE_OPENAI_EMBEDDING_MODEL=WhereIsAI/UAE-Large-V1

[bakongi]

You should create new generator file and should not use openai library. Use together python library instead.

https://docs.together.ai/docs/inference-python

Anyway, it doesn't work from the box. https://docs.together.ai/docs/embedding-openai-api-compatibility

[thomashacker]

Hey, thanks for the issue! Make sure that you don't have a OPENAI_API_KEY global environment variable, since it will overwrite any local .env file. Either check for global variables, or use source .env before starting verba. Let me know if this helps 🚀