InferenceMAX
InferenceMAX copied to clipboard
Use vLLM framework for DeepSeek R1 on MI325 and MI355 hardware
Please consider changing framework for DeepSeek R1 to vLLM, it shows better performance over SGLang. Here is also documentation for running DeepSeek with vLLM.
@cquil11 , @functionstackx, I don't have the permission to assign a reviewer, so just tagging you both :). I know you guys are figuring out the load on the CI before moving forward with the review. Let me know what we can do to help.
@qcolombet yes, we are looking into it
@cquil11 is just trying to land an massive refactor PR first to reduce tech debt and then we can look into this one
@merrymercy