chintan-ushur
chintan-ushur
@danielhanchen -- As unsloth's one of the primary strengths lie in fine tuning and multi-gpu is one of the most important feature, I think we should prioritize this. I can...
I have the same issue, ```Error: pull model manifest: Get "Authentication%20required?nonce=ezvYCLJ-1x8BeUB3BkZWnQ&scope=&service=&ts=1731390067": unsupported protocol scheme ```
@Isotr0py -- I tried with GPU backend, the error persists, please advise: `OSError: It looks like the config file at '/mnt/.cache/huggingface/hub/models--TheBloke--TinyLlama-1.1B-Chat-v1.0-GGUF/snapshots/52e7645ba7c309695bec7ac98f4f005b139cf465/tinyllama-1.1b-chat-v1.0.Q4_0.gguf' is not a valid JSON file.`
The solution provided by @Isotr0py works well for me.
Same issue, vllm==0.6.4 and torch==2.5.1
The fix is in progress, meanwhile here is a quick workaround: `patchelf --add-rpath '$ORIGIN/../../nvidia/nvjitlink/lib' /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.so ` Ref - [https://github.com/pytorch/pytorch/issues/140797](url)
Same here: Key issue appears to be: `GGML_ASSERT((qs.n_attention_wv == n_attn_layer) && "n_attention_wv is unexpected") failed` -- Update: The model was not downloaded correctly, I re-ran "`python setup_env.py --hf-repo HF1BitLLM/Llama3-8B-1.58-100B-tokens -q...