Yao Dong

icon company Institute of Information Engineering, Chinese Academy of Sciences icon location Beijing, China icon location Ph.D. Graduate, University of Chinese Academy of Sciences. Author of YAYI 2 Technical Report. Beijing Wenge Technology Co., Ltd.

Results 1 comments of Yao Dong

使用zero3_offload+序列并行训练yi-34b的时候出错

这应该是deepspeed0.14.1的一个bug，0.14.0不会在offload的时候出问题，deepspeed issue有人讨论