AssertionError: You can't use same `Accelerator()` instance with multiple models when using DeepSpeed

[well0203]

Hi, thank you for your work again. I have another question: How did you train models with multiple iterations if "AssertionError: You can't use same Accelerator() instance with multiple models when using DeepSpeed." I set --itr 2 and got this error. I am wondering, how you solved this issue? Did you just run main scripts one by one and then averaged results with different seeds, or you fixed it somehow in main script so that it supports multiple models/iterations? I am asking, because I did not find stable solution or fix around, because DeepSpeed currently supports only one model. Thank you in advance.