microsoft
How to start deepspeed automatically?
If there is a main.py, I want to train with deepspeed through a parameter, since the commands for these two training methods are different. Such as, if the parameter==true, training with deepspeed, else training in the normal way. Further, how to automatically choose to train with zero 1 or 2 or 3?
Hi @qwerfdsadad - you should be able to start with the docs here to see how to conditionally wrap your model with DeepSpeed.
Zero stages 1,2, and 3 come with different tradeoffs, you do not need to use them, but they are increasingly useful if you run out of memory. Docs here
Hi @qwerfdsadad - closing this as stale for now. Please comment if you need more assistance.