Bin Lin (林彬)
Bin Lin (林彬)
We add to todo list. but probably won't focus on that at the moment.
https://github.com/PKU-YuanGroup/Open-Sora-Plan/blob/main/opensora/train/train_t2v.py#L494-L502
You can calculate the corresponding input size based on https://github.com/PKU-YuanGroup/Open-Sora-Plan/issues/128.
Thank you for your advice! We've included it as a future plan.
Released in next version.
Do you enable the gan loss? We also meet it, it will happen after ~30-50k steps. But it does not matter, just resume it.
In v1.0.0 we didn't use gan loss. In v1.1.0 vae's capabilities will be vastly improved.
The default training is on zero2 mode.
Sorry, since he's not an autoregressive model, it may not be possible to generate a new frame based on the previous one. But maybe we can explore considering the previous...
Yes, we use it directly. It adapts very quickly and the transformation is visible in about 500 steps. This is consistent with [pixart-sigma](https://arxiv.org/abs/2403.04692).