Train on 256**2 and then finetune on larger resolutions---how is this done?

[VigneshSrinivasan10]

Dear Authors,

Thank you very much for your efforts and also for open sourcing your code and models.

In the README, you mentioned the following: sd-v1-1.ckpt: 237k steps at resolution 256x256 on [laion2B-en](https://huggingface.co/datasets/laion/laion2B-en). 194k steps at resolution 512x512 on [laion-high-resolution](https://huggingface.co/datasets/laion/laion-high-resolution) (170M examples from LAION-5B with resolution >= 1024x1024).

I am unclear on how do you perform the finetuning. Does it refer to the finetuning of the

1. Diffusion on the latent space
2. Autoencoder
3. Both

I would assume there would be a change in the resolution of the latent space when the input image resolutions are changed. It would help me immensely if you could clarify on this. Thanks in advance.

[CreamyLong]

The change of resolution will result in the change of latent space.

However, this change does not affect the structure of Autoencoder and Unet in the diffusion. You can see the code of Autoencoder and Unet.

By the way, I guess the authors only train the Unet model due to the frozen of first stage model