chatglm.cpp icon indicating copy to clipboard operation
chatglm.cpp copied to clipboard
Published 1 year ago • verified publisher icon li-plus

130B

Open iHaagcom opened this issue 2 years ago • 1 comments

Can this work with the GLm130B model? https://github.com/THUDM/GLM-130B

iHaagcom avatar Jul 06 '23 16:07 iHaagcom

Probably not. At least not for now. It'll be extremely slow on CPU, and it's too large to fit into a single GPU even A100-80GB. Need to support tensor parallelism and it's a lot of work.

li-plus avatar Jul 13 '23 14:07 li-plus