Kunal Suri

I haven't. Just didn't try it further. On Tue, 13 Jun, 2023, 8:15 pm sile, ***@***.***> wrote: > @suri-kunal hi, How did you fix the bug ? > Can you...

No Petros. I didn't try it further. On Sun, 11 Jun, 2023, 11:08 am Petros Karypis, ***@***.***> wrote: > Did you ever figure out a solution for this? > >...

> > As you can see, GPU Usage in ZeRO3 is increasing as compared to ZeRO2. I tried using `model_engine.empty_partition_cache()` as well but I got an error that `empty_partition_cache` attribute...

> @suri-kunal. thanks for reporting this issue. Can you please add this to the "zero_optimization" section of ds_config > > ```json > "memory_efficient_linear": false > ``` I am getting the...

> Can you please use latest deepspeed? Issue resolved!! Thanks for creating this library. Could you please look into #3377 as well?

@tjruwase, @tohtana I am sorry but the issue still seems to persist. I have even applied `model_engine.empty_partition_cache()` as well. Please the attached plot -  The 'glamorous-sweep-1'...

> @suri-kunal, thanks for sharing this update. We will investigate further. Any update on this issue?