BitNet icon indicating copy to clipboard operation
BitNet copied to clipboard
Published 8 months ago • verified publisher icon microsoft

Does it support distributed inference? I want to deploy large models on an XPU cluster. like cpu cluster

Open kotaking opened this issue 10 months ago • 3 comments

Does it support distributed inference? I want to deploy large models on an XPU cluster. like cpu cluster

kotaking avatar Apr 21 '25 01:04 kotaking

+1

baonq-me avatar Apr 22 '25 07:04 baonq-me

what size of model are you talking about?

sd983527 avatar Apr 24 '25 08:04 sd983527

deepseek 32B

kotaking avatar May 07 '25 02:05 kotaking