stable-diffusion-webui icon indicating copy to clipboard operation
stable-diffusion-webui copied to clipboard

Protect alphas_cumprod during refiner switchover

Open drhead opened this issue 2 years ago • 0 comments

Description

  • There is currently a bug, mentioned in #14978, where when the refiner switches on, the first step it performs uses the original alphas_cumprod schedule, which causes problems if zero SNR is enabled.
  • This stores and re-applies the model alphas_cumprod when refiner switchover happens. This fixes outputs in rare cases where the change in noise schedules is significant enough to change the called timestep to something outside of the range of the refiner.

Screenshots/videos:

Before fix, image generated on DPM++ 2M, overridden with Karras schedule and sigma_max of 1500, 50 steps: 99942-36463525-(best quality, high quality,_1 5) by strange-fox, solo, anthro, male, rat, manly, clothed, jacket, shirt, detailed background, n This specific schedule causes one of the sampling steps to be changed from timestep 190 to timestep 200. The highest timestep a typical refiner is trained for is 199 (last 200, zero indexed), so this is out of the range for the model and causes extra noise in the output.

After the fix is applied: 99943-36463525-(best quality, high quality,_1 5) by strange-fox, solo, anthro, male, rat, manly, clothed, jacket, shirt, detailed background, n The resulting image looks much cleaner (particularly the background at the top left).

Checklist:

drhead avatar Feb 21 '24 00:02 drhead