Song

[BUG] Fail to inference with 8bit quantized bloom-3b model, shape mismatch error

10

**Describe the bug** Hi, I am trying to run a bloom3b model with 8bit quantization with two 2080Ti GPUs. Because there exists no pre-quantized version, I simply change the `dtype`...

[New Model]:Step Audio 2

11

### The model to consider. https://github.com/stepfun-ai/Step-Audio2 ### The closest model vllm-omni already supports. qwen3 omni ### What's your difficulty of supporting the model you want? _No response_ ### Use case...

[New Model]: Step Audio EditX

8

### The model to consider. https://huggingface.co/stepfun-ai/Step-Audio-EditX ### The closest model vllm-omni already supports. _No response_ ### What's your difficulty of supporting the model you want? _No response_ ### Use case...