ColumbusAI

Can you please help?

## **Describe the bug**  MC would not connect to Ollama's localhost API endpoint so I started up liteLLM and it's working as an intermediary proxy for...

Is Support for the DeepSeek v2.5 model on the roadmap?

1

The new DeepSeek v2.5 model is the frontier model for local LLM coding. Is this on the roadmap for support?

Enhancement ideas

1

Just sharing some I think would be impactful- - tooltips for hotkeys - ability to change hotkey bindings (for ex. i run into issue on windows using Ctrl-R to regenerate...

[BUG] 'merge_4bit_forced' missing custom logic to save 4-bit dynamic quant

1

I'm running the GRPO code locally and if I leave in the default- ``` # Merge to 4bit if True: model.save_pretrained_merged("model", tokenizer, save_method = "merged_4bit",) ``` Then I end up...

[FEATURE] Sharded Quantization to Enable Consumer GPUs to Quantize 80B+ LLMs

4

Hey guys, can you please consider introducing sharded quantization as a core feature? This would allow users to quantize extremely large models (e.g., _80B+ parameters... CohereForAI/c4ai-command-a-03-2025_!) on consumer-grade hardware by...

[DOC] All of the Quantization Examples have wrong reference to `backends`.

1

[Scripts referenced in this readme](https://raw.githubusercontent.com/ModelCloud/GPTQModel/refs/heads/main/examples/README.md) There are only a couple of scripts:  Compared to the readme: