FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard
verified publisher icon lm-sys

Plans to support Gemma 3?

Open deep1401 opened this issue 10 months ago • 1 comments

Are there any plans or is anyone working on integrating support for the Gemma 3 models within Fastchat? Natively loading a Gemma 3 model doesn't directly work and opens up loading as well as inference issues.

deep1401 avatar Mar 19 '25 16:03 deep1401

Perhaps it would make sense to publishing’s bug on SGLang, as the model_worker is not really where you want to run these models?

surak avatar Mar 20 '25 09:03 surak