[Pipeline] ReNoise: Real Image Inversion Through Iterative Noising

[clarencechen]

Model/Pipeline/Scheduler description

Achieving faithful image-to-noise inversion with Denoising Diffusion models remains a challenge, particularly for more recent models trained to generate images with a small number of denoising steps. This work introduces an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. Building on reversing the diffusion sampling process, the method applies multiple fixed-point iterations to estimate the next inversion target at each noise level, and then averages the predictions to empirically increase image reconstruction quality. Furthermore, this method preserves editability through optimizing noise regularization losses in a fashion similar to Pix2PixZero.

Open source status

• [ ] The model implementation is available.
• [ ] The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

• Arxiv PDF: https://arxiv.org/pdf/2403.14602.pdf
• Github: https://github.com/garibida/ReNoise-Inversion
• First Author: @garibida
• Notes: The code is not available yet, but the method should be easy to implement as a new pipeline similar to the existing StableDiffusionPix2PixZeroPipeline based on the pseudocode and discussions in the paper.

[satani99]

hey @sayakpaul i'd like to work on this. can you assign this to me?

[sayakpaul]

Sure, feel free to start a PR :)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[garibida]

Hi, you can use my code as reference: https://github.com/garibida/ReNoise-Inversion

[satani99]

thanks @garibida

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.