gather_norm issue

[YanzuoLu]

As the comments I write here, I'm wondering is the gather_norm settings during training are identical to the configs given in this repo?

I see that CogVideoX-1.0(2B & 5B) is using gather_norm for encoder only, so that the decoder can be run in frame-wise manner.

But in CogVideo-1.5 it seems that both encoder and decoder are using gather_norm, does it indicate that framewise decoding is always at loss and cannot be aligned?

Thanks for your time.

[a-r-r-o-w]

Sorry for the delay! I have responded in the diffusers thread

[YanzuoLu]

@zRzRzRzRzRzRzR Any updates on this?