DiffSynth-Studio icon indicating copy to clipboard operation
DiffSynth-Studio copied to clipboard

EntityControl 是否支持dreambooth Lora?

Open JoshonSmith opened this issue 11 months ago • 3 comments

great work!

https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/EntityControl/entity_control.py

请问EntityControl是否在加载自己的lora权重之后,再加载dreambooth Lora,实现个性化的区域控制生成?

JoshonSmith avatar Mar 03 '25 05:03 JoshonSmith

从原理上讲,EliGen 的LoRA是能和其他Lora共用的,比如IP-Adapter和In-Context Lora,目前我们只对这两个进行了适配,详情可以参考代码。请问您具体想用的dreambooth Lora是哪个呢?可以发一下具体的模型链接,我们会跟进适配!

mi804 avatar Mar 03 '25 06:03 mi804

I’m also interested in applying a DreamBooth LoRA with Eligen. Were you able to successfully load the DreamBooth LoRA trained with Diffusers? As noted in #381, the current version of DiffSynth does not support this lora format. A lora model link was provided there as well.

wingvortex avatar Mar 03 '25 13:03 wingvortex

@wingvortex @JoshonSmith We have supported the dreambooth lora. Please see examples/EntityControl/styled_entity_control.py for example. Here are the visualizations:

Image

Since the parameters contained in different LoRAs are not exactly the same, this may not be a one-size-fits-all solution. If there are any other compatibility issues, please let us know.

mi804 avatar Mar 05 '25 03:03 mi804