huggingface
Stable Cascade Controlnet training
Is your feature request related to a problem? Please describe. Last year I wrote a long article on how to train controlnets using diffusers, and trained two controlnets using diffusers. A few months ago Stable Cascade was released which requires less resources than SDXL and also should make controlnet training for high quality diffusion models more viable. I tried to run the official training script (batchsize=4, bf16) but ran into OOM. I'd hope that a diffusers implementation would provide more optimization and lower VRAM requirements.
Describe the solution you'd like.
Please provide a examples/controlnet/train_controlnet_stablecascade.py
Describe alternatives you've considered. The official training script => OOM
Additional context. A training script for the Würstchen architecture was already anticipated a long time ago (see https://github.com/huggingface/diffusers/issues/5071).
Pretty please!
Sorry, we won't have the bandwidth to work on that script right now. I will leave it open to the community in case someone from the community wants to pick it up.
Hi @sayakpaul, could you pls tell how may I start working on this? Like how may I proceed in doing such tasks? Any help will be appreciated. Thanks
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.