lllyasviel
Masked diffusion example
In the paper, there is the masked diffusion example (Fig. 16). I see that in the sampling functions the (eg. ddim_sampling function) mask image and the initial image (x0) are also input parameters but their default values are None. I couldn't make it work when I want to use a mask image and an initial image. Please provide an example script of masked diffusion usage. Specifically, how do we prepare the x0 , x_T and mask?
On this note, how would someone go about using both the original image unmodified + some other control like canny (using img2img instead of txt2img for the base SD). I'd like to preserve some color information that canny, hed, m-lsd, etc remove
Looks like ldm module is similar to original SD code https://github.com/CompVis/stable-diffusion/tree/main/ldm. I am assuming the original img2img script should just work https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py. But still some code changes are required.
Hi guyes, @himmetozcan when you say, you mean documentation right? Could you share whit the url? i'm new in stable diffusion and controlnet, i would like to understand how works each parameters. Thanks.
Do we have any updates for masked diffusion example? I think there's several options for inpainting or masked diffusion? Do we need to resize the x0 and mask to h//8 ,w//8 since it's LDM, but the channel number is unmatched, the random img has 4 channels, rather than 3. Please shed some lights for us. Thanks.
I got background composition working by using the "encode_first_stage" function to encode the background image, then passing encoded background tensor as well as a mask to the "sample" function. Code snippet below:
