DaBossCoda

Results 8 comments of


                                            DaBossCoda

Recent vLLMs ask for too much memory: ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

same here

Feature request：support ExLlama

Also looking forward to exllamav2 support

memory leak

Also, having a memory leak issue. Can't seem to fix it :/

[Usage]: Flash Attention not working any more

Same

[Bug]: VLLM 0.8.2 OOM error (No error in 0.7.3 version)

+1

[Bug]: sm75 can not serve qwen3 bnb 4bit model

how do we use qwen 3 moe quantized?

[Bug]: sm75 can not serve qwen3 bnb 4bit model

?

[Bug]: RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

same here using v1