武帅

icon indicating a website link https://github.com/cainiaoup

icon company The University of Hong Kong icon location Hong Kong icon location A PhD student at HKU with research interests in LLM and Reinforcement Learning.

Results 1 issues of 武帅

About multi cards inference.

the code of "inference/infer_single.py" can not infer "TRACE-Benchmark/LLM-CL-Benchmark_1000", due to CUDA OOM(A100 8G) and only can work on the single card. it's not reasonable. I think the code of "inference/infer_multi.py"...