DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard
verified publisher icon microsoft

[REQUEST] mpu module

Open Ant0082 opened this issue 3 years ago • 0 comments

Is there an API for the MPU? Where can I find how to use these methods? Do I need to experiment one by one manually?

mpu.copy_to_model_parallel_region
mpu.gather_from_model_parallel_region

Ant0082 avatar Feb 14 '23 03:02 Ant0082