diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard
Published 2 months ago • verified publisher icon huggingface

Request support for MAGI-1

Open lavinal712 opened this issue 8 months ago • 8 comments

Model/Pipeline/Scheduler description

MAGI-1 is a video generation model that has achieved stunning visual effects.

Open source status

  • [x] The model implementation is available.
  • [x] The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

https://github.com/SandAI-org/MAGI-1 https://github.com/SandAI-org/MagiAttention

lavinal712 avatar May 08 '25 03:05 lavinal712

Hi, just checking in to see if there are any plans to integrate MAGI-1 into diffusers. I’d love to use it within the library. Any updates or thoughts on this would be great to hear!

kuantuna avatar May 19 '25 11:05 kuantuna

it is not diffusion model so we won't be supporting it for now

yiyixuxu avatar May 19 '25 16:05 yiyixuxu

I think it use diffusion method (or flow matching) in sampling. Am I wrong?

lavinal712 avatar May 19 '25 18:05 lavinal712

@yiyixuxu It is essentially a flow matching model but different chunks of frames are generated autoregressively once the previous chunk reaches a certain noise threshold. If we wanted to compare against a model that was similar to MAGI, I would say it's like Framepack.

In framepack, we generate the video in chunks as well, but the denoising of each chunk starts after previous one fully finishes (the relation between past chunks and current chunk is established via the history latents).

In magi, it's somewhat similar, but the relation between current chunk with previous chunks is established with a causal mask, after previous chunks reach certain noise level during the denoising process

a-r-r-o-w avatar May 19 '25 18:05 a-r-r-o-w

ohh thanks for explaining! so it is still fundamentally a denoising process. We can look into support then if it can fit into our existing model/pipeline design

yiyixuxu avatar May 20 '25 14:05 yiyixuxu

MAGI-1 4.5B distill and distill+quant models are now released https://huggingface.co/sand-ai/MAGI-1/tree/main/ckpt/magi

nitinmukesh avatar May 25 '25 17:05 nitinmukesh

Looks like there seem to be some similarities with SkyReels-V2, too. SkyReels-V2 utilizes Diffusion Forcing to stay autoregressive, keeping things rolling with a delay between chunks (like ar_step=5, causal_block_size=5). It ties past and future chunks with partially denoised latents + causal mask.

If MAGI-1 is proper for diffusers and contributions-welcome, I'd love to work on porting MAGI-1 to diffusers.

tolgacangoz avatar Jun 08 '25 18:06 tolgacangoz

hi @tolgacangoz sounds good!

yiyixuxu avatar Jun 12 '25 19:06 yiyixuxu