stablediffusion icon indicating copy to clipboard operation
stablediffusion copied to clipboard

[Bug] The "decode_first_stage" function in DDPM does not respect the "force_not_quantize" parameter

Open wywywywy opened this issue 3 years ago • 2 comments

The decode_first_stage function in the ldm/models/diffusion/ddpm.py file looks like this.

    def decode_first_stage(self, z, predict_cids=False, force_not_quantize=False):
        if predict_cids:
            if z.dim() == 4:
                z = torch.argmax(z.exp(), dim=1).long()
            z = self.first_stage_model.quantize.get_codebook_entry(z, shape=None)
            z = rearrange(z, 'b h w c -> b c h w').contiguous()

        z = 1. / self.scale_factor * z
        return self.first_stage_model.decode(z)

The predict_cids and force_not_quantize parameters are accepted but never used.

The last line in the old repo looks like this, which makes more sense: return self.first_stage_model.decode(z, force_not_quantize=predict_cids or force_not_quantize)

So the question is, will it break anything applying this change?

wywywywy avatar Nov 29 '22 22:11 wywywywy

is this why it uses SO MUCH VRAM to decode? Above 2k isn't possible....so the new upscaller can't even be used on the 768 model.

richservo avatar Nov 30 '22 03:11 richservo