Jason Clarence
Jason Clarence
Well yeah, idk i think it would be great to use the deps from that repo
+1, looking to figure this out soon
why is this not merged yet 😊😊
Yeah worth documenting this on the usage examples maybe
I don't get what you mean, maybe you mean wanna add something to the vllm for your own use? maybe you can try making a dockerfile but with the base...
yeah i guess now we dont need set quantization env var but we need to support it like from this one 
 Pls support new quantization fp8, refer to this docs: [vllm docs](https://docs.vllm.ai/en/latest/quantization/fp8.html) I've got a whole new menu with a bunch of new options i guess its all of the...
Duplicate issue with: this #83
> @TimPietrusky #82 is updated. I am currently doing sanity before it's released fully. In case, If you want to try it out. You can build an image and deploy...
oh wow, okay thanks for the info~! didn't know that was possible