basaran
basaran copied to clipboard
hyperonym
Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.
Updates the requirements on [transformers[sentencepiece]](https://github.com/huggingface/transformers) to permit the latest version. Release notes Sourced from transformers[sentencepiece]'s releases. v4.37 Qwen2, Phi-2, SigLIP, ViP-LLaVA, Fast2SpeechConformer, 4-bit serialization, Whisper longform generation Model releases Qwen2...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Updates the requirements on [accelerate](https://github.com/huggingface/accelerate) to permit the latest version. Release notes Sourced from accelerate's releases. v0.26.1: Patch Release What's Changed Raise error when using batches of different sizes with...
Hello, Trying to get llama-2-7b-chat-hf working but getting this error : docker run -p 80:80 -e MODEL=meta-llama/Llama-2-7b-chat-hf hyperonym/basaran:0.21.1 Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/huggingface_hub/utils/_errors.py", line 261, in hf_raise_for_status response.raise_for_status()...
Is there a plan or an implementation of basaran for the runpod serverless environment?
Getting error: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1! I am running basaran with default params and llama...
Updates the requirements on [huggingface-hub](https://github.com/huggingface/huggingface_hub) to permit the latest version. Release notes Sourced from huggingface-hub's releases. v0.18.0: Collection API, translated documentation and more! Collection API 🎉 Collection API is now...
Can you create a Gradio app for this Trained model
If you could just pass on requests to OpenAI, you'd have a quicker way to test connecting apps.
## Checklist - [x] Regardless of whether the last role in `messages` is `user` or `assistant`, the response will always be `assistant`. - [ ] When `steam=true`, the first returned...
Hi, I'm running Basaran via Docker and I have now tried using several different models at this point but every time after its downloaded and load everything, I'm facing with...
