How can we use DPM++ 2M Karras sampler in Diffusers?

[danishboy000]

I'm trying to use the latest samplers which provide better performance with stable diffusion in the diffusers library, but I couldn't find DPM++ 2M Karras sampler in the library. These samplers are present in Automatic111. Can anyone guide me on how to use these samplers in the Diffusers library?

[patrickvonplaten]

I'm trying to use the latest samplers which provide better performance with stable diffusion in the diffusers library, but I couldn't find DPM++ 2M Karras sampler in the library. These samplers are present in Automatic111. Can anyone guide me on how to use these samplers in the Diffusers library?

Hey, this should be identical to this scheduler: https://huggingface.co/docs/diffusers/api/schedulers/multistep_dpm_solver

Could you try using this one to see if quality matches the one of Auto1111.

[danishboy000]

I tried this and the quality didn't match.

[pcuenca]

I think @danishboy000 might be referring to the use of Karras sigmas #1633 in combination with this scheduler's algorithm, but I'm not an expert in Automatic1111 nomenclature. Perhaps @apolinario can confirm?

[apolinario]

Yes that is correct!

[patrickvonplaten]

@danishboy000 could you copy-paste some code snippets that you used for Auto1111 and diffusers for comparison?

[apolinario]

@patrickvonplaten , I think in this issue https://github.com/huggingface/diffusers/issues/1633 I showcase the code snippet used by AUTO1111 for using Karras sigmas for all samplers

[Stax124]

@patrickvonplaten I was able to get KDiffusion samplers with Karras sigmas with some modifications to your KDiffusion experimental model:

https://github.com/Stax124/voltaML-fast-stable-diffusion/blob/4b6a23c5a09ee6375b6dbc86ba1d207df80eea7b/core/diffusers/kdiffusion/KDiffusionModel.py#L513

I am getting similar results to A111 (using AnythingV4 - there are multiple versions of the checkpoint, hence the tiny change, quality-wise seems good to me)

Volta:

A111:

I would make a PR but I am busy right now, so if you have someone that can take a look at it, feel free to add it. If not, I will create a PR in the future, but no idea when that will happen.

PS - There are more optimizations in the model, callback was implemented as well.

Thanks for the hard work on diffusers, we really appreciate it.

[patrickvonplaten]

Very cool, thanks a lot @Stax124 :-)

We have an open issue to add karras sigmas to diffusers, I'll link it to your work already thanks a lot :-)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[Kenshiro-28]

Is now possible to use DPM++ 2M Karras on DiffusionPipeline? I'm using custom_pipeline = "lpw_stable_diffusion"