Xiao

run the code has error

4

I use 4 gpus to run the code. my command is ``` torchrun --nproc_per_node 4 test/test_ring_flash_attn_varlen_func.py ``` my error is ``` rank1]: Traceback (most recent call last): [rank1]: File "/home/xxxx/ring-flash-attention/test/test_ring_flash_attn_varlen_func.py",...

[Bug/Assistance] how to use local model to replace gpt3.5?

3

**Describe the bug** A clear and concise description of what the bug is. **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll...

hi, could you provide a code like llama3?

2

hi, Dear DeepSeek team: if your team could provide a implementation like https://github.com/meta-llama/llama3/tree/main/llama, it will be great. Llama3 is very helpful to learn.

my code ``` model_name_or_path = "meta/llama2-3.1B" mii_engine = mii.pipeline(model_name_or_path = args.model_type, tensor_parallel = 1) mii.engine(prompt="hello, paris") ``` the error is ``` [rank0]: deepseedfastgen_engine = get_engine(args, arrival_rate) [rank0]: File "/root/xxxx/DeepSpeed-MII/hybridserve/serve.py", line...

[Bug] how to set the args.predictor_param_file in python/sglang/srt/bullet/shared_mng.py

2

### Checklist - [x] 1. I have searched related issues but cannot get the expected help. - [x] 2. The bug has not been fixed in the latest version. -...