language-model-arithmetic icon indicating copy to clipboard operation
language-model-arithmetic copied to clipboard
verified publisher icon eth-sri

Inference acceleration

Open CQUPT-CaiKe opened this issue 8 months ago • 1 comments

Excellent work. I would like to know if it is possible to use some commonly used inference acceleration frameworks such as VLLM and LMDEPLOY in the model loading section.

CQUPT-CaiKe avatar Jun 10 '25 08:06 CQUPT-CaiKe