InferenceMAX icon indicating copy to clipboard operation
InferenceMAX copied to clipboard

Use vLLM framework for DeepSeek R1 on MI325 and MI355 hardware

Open omirosh opened this issue 3 months ago • 3 comments

Please consider changing framework for DeepSeek R1 to vLLM, it shows better performance over SGLang. Here is also documentation for running DeepSeek with vLLM.

omirosh avatar Oct 14 '25 13:10 omirosh

@cquil11 , @functionstackx, I don't have the permission to assign a reviewer, so just tagging you both :). I know you guys are figuring out the load on the CI before moving forward with the review. Let me know what we can do to help.

qcolombet avatar Oct 17 '25 15:10 qcolombet

@qcolombet yes, we are looking into it

@cquil11 is just trying to land an massive refactor PR first to reduce tech debt and then we can look into this one

functionstackx avatar Oct 17 '25 17:10 functionstackx

@merrymercy

functionstackx avatar Oct 22 '25 19:10 functionstackx