lyy-zz

Results 2 comments of lyy-zz

[PROPOSAL]: Can you write down the detailed steps and technical details of the second and third steps?

> Hi, thanks for your interest. > > Yes, we are planning to release a technical report of our training process. Please stay tuned. Thanks Will the technical report or...

[Usage]: Not enough memory when run a 33b model float16 on 2 x L40 GPU (48G)

deploy qwen1.5_72b_chat on two A800 GPUs (80G) with vllm 0.4.2, also getting the error ``` CUDA_VISIBLE_DEVICES=2,3 python -m vllm.entrypoints.openai.api_server \ --model Qwen1.5-72B-Chat \ --dtype auto \ --gpu-memory-utilization 0.9 \ --tensor-parallel-size...