DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Add chatglm2 & chatglm3 autotp

Open Yejing-Lai opened this issue 1 year ago • 0 comments

This PR aims to enable chatglm2 & chatglm3 autotp. Similar to the phi3, this model uses the chunk MLP layer, so we adjust the weight order by 'shard_mlp_chunk' func. Please kindly review~ Thanks!

Yejing-Lai avatar May 16 '24 04:05 Yejing-Lai