DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

[REQUEST] Support profiling backward using Flops Profiler as a standalone package

Open yzs981130 opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe. I try to use flops profiler as a standalone package to profiling the backward of a custom model. I have followed both two instructions, here and here. However, both examples only consider profiling the forward stage. I check the code and it seems that it currently only prints the bwd_latency when using runtime as below https://github.com/microsoft/DeepSpeed/blob/5e42cc8be22f58ef6be022fcc6d69fe11c7bfe25/deepspeed/profiling/flops_profiler/profiler.py#L313-L317

Describe the solution you'd like Is there any approach to profiling the backward by the flops profiler?

yzs981130 avatar Aug 18 '22 02:08 yzs981130