VAE config mismatch when using StableDiffusionInpainting

[sayakpaul]

Currently, we have got a failing test:

tests/pipelines/stable_diffusion/test_stable_diffusion_inpaint.py::StableDiffusionInpaintPipelineSlowTests::test_single_file_component_configs

And it fails for the right reason! You will notice that the assertion doesn't pass for sample_size. For the diffusers checkpoint, this value is 256 and the inferred value from the single-file checkpoint is 512.

This is a bit of an issue because the sample_size param in the VAE influences the tiling factors: https://github.com/huggingface/diffusers/blob/5d848ec07c2011d600ce5e5c1aa02a03152aea9b/src/diffusers/models/autoencoders/autoencoder_kl.py#L119

So, for the VAE loaded from the diffusers checkpoint will have a different tiling configuration from the one loaded from the single-file checkpoint.

Cc: @DN6 @yiyixuxu

[sayakpaul]

gentle ping @yiyixuxu

[yiyixuxu]

I think this will not be an issue once we have https://github.com/huggingface/diffusers/pull/7496, no?

[sayakpaul]

I will let @DN6 test that.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.