where is pretrain_zeropp_gpt.py

Open guoyejun opened this issue 2 years ago • 5 comments

hi, pretrain_zeropp_gpt.py is mentioned in zeropp tutorials at https://github.com/microsoft/DeepSpeed/blob/f8551b439ea4a849ecb37d148e3bacbffd64121f/docs/_tutorials/zeropp.md?plain=1#L54 , could you share where to get the file? thanks. I could not find it under https://github.com/microsoft/Megatron-DeepSpeed.

Jun 29 '23 02:06 guoyejun

@HeyangQin could you please let us know regarding pretrain_zeropp_gpt.py ? Thanks

Jul 11 '23 03:07 abhilash1910

@HeyangQin I need pretrain_zeropp_gpt.py, thanks

Oct 08 '23 09:10 etoilestar

any news on this? thanks

Jul 10 '24 16:07 puppetm4st3r

I want it, too.

Aug 16 '24 02:08 deadfffool

May 06 '25 08:05 SeekPoint