diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard
Published 2 months ago • verified publisher icon huggingface

Implement v-prediction in all schedulers

Open pcuenca opened this issue 3 years ago • 3 comments

This would be nice to guarantee scheduler interoperability in Stable Diffusion 2. DDIM, DDPM and EulerDiscrete already support v-prediction, the following ones remain (from the list of compatible stable diffusion schedulers):

  • [ ] EulerAncestralDiscreteScheduler
  • [ ] PNDMScheduler
  • [ ] LMSDiscreteScheduler
  • [x] DPMSolverMultistepScheduler

pcuenca avatar Nov 25 '22 10:11 pcuenca

I will take a stab at the DPMSolverMultistepScheduler scheduler

kashif avatar Nov 25 '22 14:11 kashif

@pcuenca would be interested in contributing, but not sure what v-prediction is. Could you give me a summary / resources?

vvvm23 avatar Nov 30 '22 10:11 vvvm23

thank you for your interest @vvvm23

So v-prediction refers to a type of loss objective in diffusion models. It is somewhat technical but perhaps check Section 4 of "Progressive Distillation for Fast Sampling of Diffusion Models" paper https://arxiv.org/pdf/2202.00512.pdf and let me know if you have further questions...

kashif avatar Nov 30 '22 11:11 kashif

Closing this one as it's tackled here: https://github.com/huggingface/diffusers/pull/1505

patrickvonplaten avatar Dec 01 '22 17:12 patrickvonplaten