ChatGLM-6B icon indicating copy to clipboard operation
ChatGLM-6B copied to clipboard
verified publisher icon THUDM

[Feature] 是否支持多主机并行？

Open sharlec opened this issue 3 years ago • 2 comments

Is your feature request related to a problem? Please describe.

GLM-130B的官方文档里支持使用FasterTransformer实现多主机多卡, 同样的步骤是否能应用在ChatGLM-6B上？Int-4的版本呢？

Solutions

Additional context

No response

sharlec avatar Mar 23 '23 15:03 sharlec

同问，想实现单机多卡/多机多卡上运行

imClumsyPanda avatar Mar 24 '23 02:03 imClumsyPanda

多机多卡不适合通常的inference场景，因为多机之间的通信带宽通常无法满足延时要求，所以不会支持

duzx16 avatar Apr 12 '23 15:04 duzx16