Issue: Update VLLM to Version .5.0++, and a few suggestions

[nerdylive123]

Description

1. 🌟 Upgrade VLLM: We need to rocket VLLM to version 0.5.0++ or beyond! 🚀
2. 🤖 Tensorize Awesomeness: The tensorize feature is like giving VLLM a turbo boost. 🏎️ Check out the Tensorize VLLM example for a sneak peek.
   • 🚀 It lets us load the model during download (but remember, the model needs a little conversion magic).
3. 📦 Pip It Up: Why build VLLM from scratch when we can summon it with a pip package? Efficiency, my friend! 🧙‍♂️

Kudos to the stellar maintainer! 🌟🙌

[FrederikHandberg]

+1! I really would like to run Phi3VForCausalLM

[Sapessii]

+1!

[shivanker]

+1, Gemma 2 support has been recently rolled out in vLLM!

[avacaondata]

+1, it would make much more sense to pip install vllm so that when a new model is released and implemented in vLLM it is automatically integrated in this worker @alpayariyak

[d4rk6un]

Are there any plans to upgrade the VLLM version and if so, can you provide a date?

[PhoenixSmaug]

+1, then we could finally run DeepSeek-Coder v2

[harshal-pr]

+1

Llama 3.1 needs 0.5.3 https://github.com/vllm-project/vllm/releases/tag/v0.5.3

Can we upgrade this worker to support this out of box in runpod serverless vllm ?

[Lhemamou]

waiting also for the update :) let me know if i can help !

[alpayariyak]

Hi all, thank you so much for the suggestions! I've joined a different company, so @pandyamarut will be taking over. It's been a great pleasure serving you all!

[Lhemamou]

I wish you an amazing next work experience ;) welcome aboard @pandyamarut !

[pandyamarut]

Working on it ,Sorry for the delay. Thanks for maintaining the repo @alpayariyak

[TheAlexPG]

Guys, do we know anything about the approximate time frame for the update? So that you can somehow plan the update of the models in the roadmap. Thanks

[nerdylive123]

Pls support new quantization fp8, refer to this docs: vllm docs

I've got a whole new menu with a bunch of new options i guess its all of the arguments thats very great thank you for the update staffs and maintainers! just the options value needs to be updated :)