nullnuller

Hi, Any updates on this? I am getting a models/gpt-4o is not found when using gemini with google OAI-compatible base url in magentic since it's sending gpt-4o despite setting the...

> I'm currently working on the community packagers for external LLMs through the OpenAI client [@nullnuller](https://github.com/nullnuller) - I'll keep the conversation updated @vballoli just wondering if the community package have...

> pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git@fb77505f8429566f5d21d6ea5318c342e8a67991" I also tried your suggestion but it still fails ``` ./llama.cpp/llama-cli -m model/unsloth.Q8_0.gguf -p "Below is an instruction that describes a task, paired with...

> %%capture > # Mount Google Drive for larger data sets / output > from google.colab import drive > drive.mount('/content/drive') > > !pip install --upgrade --force-reinstall "transformers==4.44.2" "numpy==2.0.2" # https://github.com/unslothai/unsloth/issues/1062...

Any progress on allowing speculative decoding in the server?

Vote for docker compose

Similar question. Is there any guide on this?

I still have the same issue https://github.com/av/harbor/issues/60#issuecomment-2411864309 The issues seems to be resolved for me. Also, I noticed that perplexica config survives rm -rf harbor or even harbor config reset