DiffEngine Plan in 2023

[okotaku]

If you have any other suggestions, please leave comments!

Fix library cores

• [x] Support new config style of MMEngine. example. https://github.com/open-mmlab/mmengine/pull/1071 https://github.com/okotaku/diffengine/pull/121

Support new methods

• [x] ControlNet
• [x] Multiple input size training like SDXL and NovelAI Aspect Ratio Bucketing. https://github.com/okotaku/diffengine/pull/50
• [x] LoRA for Convolution Layers like LyCORIS. https://github.com/okotaku/diffengine/pull/92
• [x] ControlNet Small / Mid. https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0-small https://github.com/okotaku/diffengine/pull/49
• [x] Distill SD training. https://github.com/okotaku/diffengine/pull/67
• [x] Instruct Pix2Pix. https://github.com/okotaku/diffengine/pull/89
• [x] T2I Adapter. https://github.com/okotaku/diffengine/pull/64
• [x] IP Adapter https://github.com/okotaku/diffengine/pull/59
• [ ] Training Diffusion Models with Reinforcement Learning https://huggingface.co/blog/trl-ddpo
• [x] SSD-1B: A Leap in Efficient T2I Generation https://github.com/okotaku/diffengine/pull/83

Support Datasets

• [x] Google DreamBooth. https://github.com/okotaku/diffengine/pull/52
• [ ] web dataset. https://huggingface.co/docs/hub/datasets-webdataset

Create new LoRA

• [x] Classic Anime Expressions with SDXL. https://github.com/okotaku/diffengine/pull/46 https://github.com/okotaku/diffengine/pull/48

Create new DreamBooth

• [x] Human face with Distill SD.