VLLM Serverless Endpoint Deployment using Python

[arianyambao]

Hello How do we deploy a VLLM Serverless endpoint using Python?

Do we explicitly have to create templates first? Not straight to the point of using the vLLM ready template?

[nerdylive123]

I don't get what you mean, maybe you mean wanna add something to the vllm for your own use? maybe you can try making a dockerfile but with the base image of VLLM-worker, and modify the entrypoint or CMD

[deanq]

@arianyambao Have you looked at this article? How to run vLLM with RunPod Serverless