diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard

[docs] Scheduler features

Open stevhliu opened this issue 1 year ago • 6 comments

Continuation of #7817 (see comment here) that refactors scheduler features for inference to their own doc. It includes:

  • custom timesteps and sigmas showcasing AYS
  • Karras sigmas
  • rescale_betas_zero_snr

stevhliu avatar May 20 '24 21:05 stevhliu

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @Beinsezii here we are adding a general doc page for the scheduler now; anything else we should add here?

yiyixuxu avatar May 20 '24 22:05 yiyixuxu

I feel like a section on the timestep spacings would be beneficial, especially since they're part of the same paper referenced. The paper recommends trailing which is what I and a lot of others have settled on. trailing is unique in that its mutually exclusive with steps_offset ≥ 1 as well.

Beinsezii avatar May 20 '24 23:05 Beinsezii

a good demonstration of the current generation of models' two primary forms of residual noise would probably be a good idea though i can't think of how to integrate that. i just see it a lot and i think the community needs language to describe it with, and common solutions to try. probably for a separate doc

bghira avatar May 20 '24 23:05 bghira

Thanks for the feedback, added a new section for timestep spacing!

a good demonstration of the current generation of models' two primary forms of residual noise

Good idea, maybe we can explore this in a separate PR :)

stevhliu avatar May 21 '24 19:05 stevhliu

Maybe a "Generation Quality" doc that has a bunch of common footguns. Like using Karras sigmas on models that weren't trained for it, or turning off set_alpha_to_one/final_sigmas_type.

Also I think solver order be explored in more depth either here or another doc because the best one is highly dependent on the rest of the params. Like, if you're going run 50 steps anyways a 1st order sampler will have plenty strong enough prediction with less hallucinations. Really have to balance the steps/guidance/order for your intended effect to bring out the best image rather than just bigger number better.

Beinsezii avatar May 21 '24 20:05 Beinsezii

@Beinsezii

for this, if you are able to contribute a doc we would be so grateful!

so I think solver order be explored in more depth either here or another doc because the best one is highly dependent on the rest of the params. Like, if you're going run 50 steps anyways a 1st order sampler will have plenty strong enough prediction with less hallucinations. Really have to balance the steps/guidance/order for your intended effect to bring out the best image rather than just bigger number better.

yiyixuxu avatar May 28 '24 18:05 yiyixuxu