Does the Unet have inpainting ability?

[CharlesGong12]

Hi, I am reading your excellent work and have a question. Is the unet you use a classical T2I unet or an inpainting unet?

[RunpuWei]

It seems to load the sd-base weights

[CharlesGong12]

@RunpuWei Thanks a lot but actually I mean whether the unet branch has inpainting ability before being equipped with Brushnet branch.

[RunpuWei]

@RunpuWei Thanks a lot but actually I mean whether the unet branch has inpainting ability before being equipped with Brushnet branch.

I believe the sd-base model has the capability for inpainting (e.g., Repaint: https://github.com/huggingface/diffusers/blob/bbd2f9d4e9ae70b04fedf65903fd1fb035437db4/examples/community/stable_diffusion_repaint.py), although the results were relatively poor. Additionally, you can enhance it through any fine-tuning method, such as ControlNet, BrushNet, or adding 5 additional input channels (sd-inpaint model) like runwayml.