Add support for Groq LLM inference

Open Vasanthengineer4949 opened this issue 1 year ago • 2 comments

All of the OpenAI products are paid and Ollama can be slow so please try to provide integration for Groq API to run Mixtral.

Apr 01 '24 06:04 Vasanthengineer4949

I tred to do that. but I'm getting a lot of error.

May 09 '24 15:05 achoozachooz

have you tried setting GROQ_API_KEY locally and leaving it out of the arguments?

May 14 '24 11:05 Merlinvt