diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard

CogVideoX I2V: Missing guard-rail on num_frames

Open tin2tin opened this issue 1 year ago • 4 comments

Describe the bug

CogVideoX I2V: There is a warning when the negative_prompt is defined, but no warning when num_frames is defined, however, the latter will cause a faulty render, so maybe a guard-rail warning should be added to a defined num_frames value.

The faulty video: https://github.com/user-attachments/assets/5d3b4dd9-c9d2-43bb-b14c-bdd34b490bab

Reproduction

Just add a defined num_frames value to the inference code.

Logs

No response

System Info

Latest beta diff, Win 11

Who can help?

@DN6 @a-r-r-o-w

tin2tin avatar Sep 19 '24 06:09 tin2tin

I'm having a bit of trouble understanding what's wrong here, sorry. Do you mean to say that there should be a num_frames == 49 restriction here?

a-r-r-o-w avatar Sep 19 '24 06:09 a-r-r-o-w

Well, I had a lot of faulty renders until I realized that disabling num_frames made it work. The value might have been 48.

tin2tin avatar Sep 19 '24 07:09 tin2tin

Yes, unfortunately the I2V model doesn't support anything other than 49 frames. I missed this initially so opened a patch PR to fix. Thanks for reporting!

a-r-r-o-w avatar Sep 19 '24 07:09 a-r-r-o-w

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

github-actions[bot] avatar Oct 19 '24 15:10 github-actions[bot]

Marking as closed because does not make sense to guard the value any more because of the 1.5 checkpoints that support 81 and 161 frames as well

a-r-r-o-w avatar Nov 20 '24 02:11 a-r-r-o-w