MooreThreads
Exact settings for example videos?
https://github.com/MooreThreads/Moore-AnimateAnyone#%EF%B8%8F-examples
Also any preconditioning and post editting?
We did not perform any post editing and preconditioning. The default settings (e.g. cfg=3.5, step=30) can yield similar results.
We did not perform any post editing and preconditioning. The default settings (e.g. cfg=3.5, step=30) can yield similar results.
How about the other settings? I tried to replicate the example but it still has flickering settings -W 512 -H 768 -L 120 (pose video is 30fps and 4 seconds long) --steps 30
I'm guessing the interpolation is off somewhere. How can I get just the end video the extra width may also be causing issues.
https://github.com/MooreThreads/Moore-AnimateAnyone/assets/114336644/e586890c-a43b-44ba-9c31-cadb2f863769
Could this be the reason for the flickering?
Some weights of the model checkpoint were not used when initializing UNet2DConditionModel: ['conv_norm_out.weight, conv_norm_out.bias, conv_out.weight, conv_out.bias']
It seems the motion module not working. Can you check whether the motion module params are loaded correctly?
Could this be the reason for the flickering?
Some weights of the model checkpoint were not used when initializing UNet2DConditionModel: ['conv_norm_out.weight, conv_norm_out.bias, conv_out.weight, conv_out.bias']
Encountered the same problem, have you solved it?
Any update on this?
Unfortunately not, have given up for now.
Sort of, the error message wasn't solved but the flickering was, I simply had a '.' in front of the motion module path in the json which meant it was not used. However now I get OOM because I use a t4 15gb card on colab.
Yes colab gives a maximum of 12.7 GB system RAM to free users, however, to generate one video of length 2 sec, this module seems to consume an average of 13 GB system RAM and >14GB GPU RAM.