Ashwin Cheekati
Ashwin Cheekati
@MechCoder coud you please make the colab notebook public, I think it currently has wrong permissions and comes up with the following message when trying to open the notebook. _You...
Yes and logs in orchestrator show - Unable to determine cluster name This is for a brand new cluster
deployed the cluster in a different namespace and also tried in same namespace as operator, same result. Also tried changing the name of the cluster but has same issues as...
In case anyone else has this issue, for me it was because of Installing directly from the git Main branch. When I installed the package instead from pypi there were...
Seeing the same issue with formatting, TogetherAI with Mixtral-8x7B-Instruct-v0.1 The output is not formatted as reported above by OP I'm using the litellm proxy server. Used huggingface chat-ui and LibreChat,...
I can see the tokens streamed Individually but as well, but like OP mentioned they are displayed in chunks at a time, as if the response is being first cached...
same behaviour in LibreChat as well so it looks like it's an issue with the proxy when using TogetherAI, and happens with any model on TogetherAI
@gururise have you found a workaround to this issue or are you not using Together API's?
thanks @gururise @krrishdholakia happy to help with debugging info.
@krrishdholakia I can see that the tokens are streamed when running in curl or when running the proxy in debug mode,the chunking seems to happen when the tokens are displayed...