juniper
juniper
> A PR based on the current code will be submitted this week Will it also support Qwen2.5VL?
Amazing! tp=4 can load the model, but it looks like OOM. Anyway, I only sent one request. ``` (RayWorkerWrapper pid=31401) *** SIGFPE received at time=1747278715 on cpu 140 *** (RayWorkerWrapper...
tp=4 is normal, but tp=8 still has an error ``` Traceback (most recent call last): File "/opt/py3/bin/lmdeploy", line 8, in sys.exit(run()) File "/opt/py3/lib/python3.10/site-packages/lmdeploy/cli/entrypoint.py", line 39, in run args.run(args) File "/opt/py3/lib/python3.10/site-packages/lmdeploy/cli/serve.py",...
@CUHKSZzxy Can Qwen3-235B-A22B-FP8 be used with DP and EP? If so, can you provide an example?