Xiao comments

Repositories
Issues
Comments

Results 62 comments of


                                            Xiao

[Issue]: how to deploy the local model correctly and run the `agentchat_custom_model.ipynb`?

> Hey @lambda7xx, it doesn't look like the Ollama code files are being used, the code files in the branch need to replace your installed ones, see the files here:...

[Bug] how to set the args.predictor_param_file in python/sglang/srt/bullet/shared_mng.py

Another bug: ``` scheduler.event_loop_normal() File "/home/lambda/BulletServe/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/home/lambda/BulletServe/python/sglang/srt/managers/scheduler.py", line 772, in event_loop_normal result = self.run_batch(batch) File "/home/lambda/BulletServe/python/sglang/srt/managers/scheduler.py", line 1823, in run_batch self.tp_worker.forward_batch_generation(model_worker_batch) File...