FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

SGLang worker is obsolete since SGLang 0.3.5

Open surak opened this issue 10 months ago • 1 comments

The sglang worker expects the is_multimodal_model from sglang's utility functions, but this has been removed since 0.3.5:

Traceback (most recent call last):
  File "/p/haicluster/llama/FastChat/fastchat/serve/sglang_worker.py", line 19, in <module>
    from sglang.srt.utils import load_image, is_multimodal_model
ImportError: cannot import name 'is_multimodal_model' from 'sglang.srt.utils' (/p/haicluster/llama/FastChat/sc_venv_sglang2/venv/lib/python3.11/site-packages/sglang/srt/utils.py)

surak avatar Mar 24 '25 21:03 surak

Hey @surak, We've fixed this in our forked version. Checkout https://github.com/transformerlab/transformerlab-inference

deep1401 avatar Jul 29 '25 21:07 deep1401