Dheemanth R Joshi
Dheemanth R Joshi
The examples demonstrated indicate that compile_stage resides in pippy or pippy.compile. I could not find any one of them.
So what's the guidelines for training? should i create remote optimizers for each stage and call the step method?
Not as of now.
OK, so here is what I want to do, Obtain gradients of each layer from each rank of the stage from the pipe object, and send it to the CPUs....