vladmandic
[Feature]: Ability to choose scheduler for Kandinsky
Feature description
I'm unsure if this is on purpose (as in, due to some issue) but it appears Kandinsky has DDIM hardcoded as the scheduler https://github.com/vladmandic/automatic/blob/231ca1ac7e7ceb1655b4a0c0251198556fb1351c/modules/sd_models.py#L907 and ignores the selected sampler in the UI.
If it's not on purpose (e.g. incompatibility of some kind), it'd be nice if it were possible to select the sampler like with other architectures.
Version Platform Description
No response
not all models have concept of standardized pluggable schedulers. and supporting each model architecture separately is not what i want to do. if kandinski followed same sampler format, great. otherwise, this is a no-op.
Ah, you're right, the diffusers implementation only supports DDIM and DDPM. My bad.
basically, sdnext does not "hard code" anything for sd only, it actually looks at model members to see whats supported. e.g. if there is a compatbile text_encoder, it can run prompt attention. it can try loading embeddings. etc. if there is second text_encoder (like in sdxl), it will try to use that as well. not because its sdxl, but because it recognizes a compatbile text encoder. and if there is pluggable scheduler, it will try to configure it. but if there is none, not much sdnext can do.
It supports schedulers through pipeline.scheduler setter (as is the norm on all diffusers pipelines) - that's how Kubin implements it: https://github.com/seruva19/kubin/blob/042542274db0b64d251e2fe99112a4d5ce1dccf0/src/model_utils/diffusers_samplers.py#L173
Whether they work correctly with Kandinsky is another story...
I'll reopen as its possible at least in theory. But PRs would be welcome :)
I'll poke around a bit. Although, i should probably confirm first whether it works at all (i mean alternative samplers).
A friend tested them out in the original repo and apparently, about half of them produce messed up results (either don't converge or partially converge).
There's no point. I just tested most of the commonly used samplers myself, and nearly all of them do not work well with Kandinsky.
DDIM works (obviously), also DDPM, PNDM, UniPC, DPMSSDE, that's more or less it. About half the others do not seem to converge and produce blurry or sketchy images, and the other half produces very ugly results (chimeras, what looks like patchworks of different parts of latent space, duplications etc.).
Unsure if it makes sense to pursue this. The ones that work do seem to produce noticably different results, so maybe? In the interest of experimentation?
basically, comvis samplers work and k-diffusion samplers dont? if unipc and deis work, there might be value as they're definitely better than ddim in general, but that's about it.
closing due to no progress for 1.5y - and in general, kandinsky model did not really pick up.