vdonchev
vdonchev
> Hello, > > I have the same problem and would like to know how to resolve it. Would it be necessary to submit a feature request to the repository...
+1 Also the CLI does not work at all on the latest Windows 11 regardless of the terminal used (CMD, PowerShell, Git Bash, etc.) It just displays the last two...
I tested on a few different Windows machines. Neither codex cli nor the VS code plugin are working.
Any updates on this? Seems the vllm issue was fixed in September (in vLLM >0.6.0).
I can start working on this if no one else is already.
OK then! First order of business would be to build the GGUF file(s). Seems there is an issue with that and the latest official Transformers: ``` python convert_hf_to_gguf.py .\build\bin\Release\Qwen2.5-VL-7B-Instruct\ INFO:hf-to-gguf:Loading...
Right, so this one is a bit of a rabbit hole... I. Reverting the Qwen2.5 config files to: ` "processor_class": "Qwen2VLProcessor"` and ``` "architectures": [ "Qwen2VLForConditionalGeneration" ] ``` Produces a...
UPDATE: [A few 4-bit quants have been uploaded](https://huggingface.co/IAILabs/Qwen2.5-VL-7b-Instruct-GGUF/tree/main), including two that support online auto-repacking. The latest main looks stable with **Vulkan** **CLIP** and any model thrown at it so far....
> Hi! Excelent news, thank you very much for this! > > I was able to run the model by using code from git main on a 4 x Radeon...
> > Right, so this one is a bit of a rabbit hole... > > I. Reverting the Qwen2.5 config files to: > > ` "processor_class": "Qwen2VLProcessor"` > > and...