[Issue]: BuildIn AnimateDiff v3 not working

[DevilsCrypto]

Issue Description

In the current master and latest Dev branch the buildin version of animatediff 1.5 V3 model is not working correctly. The output of the video/gif is just random images. I also see the same issue in the V2 model.

Here the result:

Version Platform Description

06:09:17-303015 INFO     Starting SD.Next                                                                                                                                                                  
06:09:17-306406 INFO     Logger: file="/home/ubuntu/automatic/sdnext.log" level=INFO size=877971 mode=append                                                                                        
06:09:17-307368 INFO     Python 3.10.12 on Linux                                                                                                                                                           
06:09:17-315447 INFO     Version: app=sd.next updated=2024-04-25 hash=8518e146 branch=dev url=https://github.com/vladmandic/automatic.git/tree/dev                                                         
06:09:17-596637 INFO     Latest published version: de267390905335d353deab3e9ac2031432f4d305 2024-04-21T12:25:50Z                                                                                           
06:09:17-598898 INFO     Platform: arch=x86_64 cpu=x86_64 system=Linux release=6.2.0-37-generic python=3.10.12                                                                                             
06:09:17-600209 INFO     nVidia CUDA toolkit detected: nvidia-smi present

Relevant log output

06:10:44-965375 INFO     AnimateDiff load: adapter="vladmandic/animatediff-v3"                                                                                                                             
06:10:56-638991 INFO     Base: class=AnimateDiffPipeline                                                                                                                                                   
Progress  1.16s/it █████████████████████████████████ 100% 20/20 00:23 00:00 Base
06:11:22-045138 INFO     Saving: image="outputs/text/00046-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-053730 INFO     Saving: image="outputs/text/00047-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-060762 INFO     Saving: image="outputs/text/00048-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-068141 INFO     Saving: image="outputs/text/00049-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-075658 INFO     Saving: image="outputs/text/00050-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-082656 INFO     Saving: image="outputs/text/00051-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-089616 INFO     Saving: image="outputs/text/00052-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-097022 INFO     Saving: image="outputs/text/00053-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-104336 INFO     Saving: image="outputs/text/00054-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-112009 INFO     Saving: image="outputs/text/00055-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-119866 INFO     Saving: image="outputs/text/00056-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-127172 INFO     Saving: image="outputs/text/00057-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-134359 INFO     Saving: image="outputs/text/00058-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-142377 INFO     Saving: image="outputs/text/00059-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-149722 INFO     Saving: image="outputs/text/00060-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-157516 INFO     Saving: image="outputs/text/00061-realistic_vision_5_1-Woman walking on street.jpg" type=JPEG resolution=512x512 size=0                                                           
06:11:22-161582 INFO     Processed: images=16 time=25.55 its=12.53 memory={'ram': {'used': 4.32, 'total': 196.56}, 'gpu': {'used': 16.58, 'total': 21.99}, 'retries': 0, 'oom': 0}                         
06:11:23-973238 INFO     Save video: file="outputs/video/00001-realistic_vision_5_1-Woman walking on street.gif" frames=31 duration=2 loop=True size=4437078

Backend

Diffusers

Branch

Dev

Model

SD 1.5

Acknowledgements

• [X] I have read the above and searched for existing issues
• [X] I confirm that this is classified correctly and its not an extension issue

[vladmandic]

known issue, needs a much deeper dive as animatediff v3 internal architecture is very different.

[vladmandic]

v3 is now supported.