Anna Cai
Anna Cai
Thank you for the suggestion! We've confirmed that adding `_reset_cuda_mem()` solves this issue. However, we are now seeing that trial 5 consistently OOMs, even when run on its own (and...
Thank you so much! Yes, we wanted to sanity check the baseline on our setup. Thank you for pointing us towards the logs, they will indeed be helpful :)
Sorry for the delayed response - unfortunately, we've found that while `inplace=True` lets the trial 5 run continue for longer, it still ends up OOMing at around iteration 3600.