Init Image strenght issue

[brurpo]

Hi there! From the pipeline: strength (float, *optional*, defaults to 0.8): Conceptually, indicates how much to transform the reference init_image. Must be between 0 and 1. init_imagewill be used as a starting point, adding more noise to it the larger thestrength. The number of denoising steps depends on the amount of noise initially added. When strengthis 1, added noise will be maximum and the denoising process will run for the full number of iterations specified innum_inference_steps. A value of 1, therefore, essentially ignores init_image.

As stated, A value of 1, therefore, essentially ignores init_image, does not work as intended, the output image always resembles the initial image, for example, a high contrast image with a black background, will always result in an image with a black background

edit:

maybe change from line 200 to something like:

add noise to latents using the timesteps

    noise = torch.randn(init_latents.shape, generator=generator, device=self.device)
    if strength >= 1.0:
        init_latents = noise            
    else:
        init_latents = self.scheduler.add_noise(init_latents, noise, timesteps).to(self.device)

The usefulness of this is to work with both init images and no init images without having to load a different pipe.

[patil-suraj]

Hey @brurpo !

That's a good point! But as stated in the philosophy here https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines#contribution pipelines are intended to be "One-purpose-only" and stay as close as possible to the official implem. The goal with these pipelines is to present simple , readable implementation that users can tweak according to their needs, rather than one-fits-all solution.

[brurpo]

Thanks for the explanation! Its really easy to customize it for yourself if you need to anyway! Thanks again!

[leszekhanusz]

Because I think it's interesting for many people I am going to maintain a unified pipeline.

to install:

pip install diffusionui

to use:

from diffusionui import StableDiffusionPipeline

then use this pipeline for either text-to-image, image-to-image or inpainting.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.