controlled-text-generation
controlled-text-generation copied to clipboard
wiseodd
KL divergence
can you give some explanations of the KL divergence term? I am a little bit confused kl_loss = torch.mean(0.5 * torch.sum(torch.exp(logvar) + mu**2 - 1 - logvar, 1)) Thank you so much!
It is the KL divergence of two Gaussian distribution (i.e., the prior p(z) ~ N(0, 1) and the posterior q(z|h) ~ N(mu, var). See the original paper https://arxiv.org/pdf/1312.6114.pdf
