Question in paper: For ablation study "Compare ControlNet and LAControlNet"

[LokiXun]

Hi, I have a question in paper. May I ask a question

In the ablation study of "Compare ControlNet and LAControlNet". Figure7(c), the w/ ControlNet result is trained from scratch based on ControlNet architecture and proposed dataset? Or using the pretrained ControlNet weights ?

After scanning from model weights released in ControlNet, those are trained on specific conditions (like sketches), I did not know which ControlNet weights to use if appling to the image restoration task.

thanks

[0x3f3f3f3fun]

Yes, we keep the architecture of ControlNet unchanged and trained it from scratch on ImageNet1k.

[LokiXun]

Yes, we keep the architecture of ControlNet unchanged and trained it from scratch on ImageNet1k.

Got it! May I ask another question. For Ablation study of "finetuning StableDiffusion v2", how the w/o finetuning model use input image as condition to guide the generation process? I see paper list 2 different paper, which confuses me. Again, thanks for your answer!

[order-a-lemonade]

Yes, we keep the architecture of ControlNet unchanged and trained it from scratch on ImageNet1k. @0x3f3f3f3fun @LokiXun Hi, may i ask a simple question: what does "train from scratch" mean? Does it means that the parallel module of Controlnet is initializing randomly instead of copying from pretrained stable diffusion?

[jiaxiangc]

@0x3f3f3f3fun Is there are model structure difference between LAControlNet and ControlNet? Can you convert LAControlNet to diffusers.

[sunzhe09]

@jiaxiangc Did you convert to diffusers successful？