latent-diffusion icon indicating copy to clipboard operation
latent-diffusion copied to clipboard

Confusion about get_learned_conditioning()

Open SoHotLB opened this issue 1 year ago • 1 comments

Confusion about get_learned_conditioning() function https://github.com/CompVis/latent-diffusion/blob/main/ldm/models/diffusion/ddpm.py#L551

def get_learned_conditioning(self, c):
    if self.cond_stage_forward is None:
        if hasattr(self.cond_stage_model, 'encode') and callable(self.cond_stage_model.encode):
            c = self.cond_stage_model.encode(c)
            if isinstance(c, DiagonalGaussianDistribution):
                c = c.mode()
        else:
            c = self.cond_stage_model(c)
    else:
        assert hasattr(self.cond_stage_model, self.cond_stage_forward)
        c = getattr(self.cond_stage_model, self.cond_stage_forward)(c)
    return c

If AutoencoderKL was used in the first stage, then the get_learned_conditioning() function should have been c = c.sample() Why is it c = c.mode() in the code?

SoHotLB avatar Feb 27 '24 09:02 SoHotLB

hi I feel like the problems we encountered are somewhat similar https://github.com/CompVis/latent-diffusion/issues/364

yanjin340121 avatar Apr 26 '24 01:04 yanjin340121