Nils Persson

tf.sqrt in WGAN-GP gradient penalty formula throws NaNs

2

I've found in training on my own image dataset that I get an InvalidArgumentError: Nan in summary histogram. This was caused by the following [line](https://github.com/tdeboissiere/DeepLearningImplementations/blob/2163918fa2e2815aa0adf907ee56fd0897a75e9c/WGAN-GP/src/model/train_wgan_GP.py#L83): `slopes = tf.sqrt(tf.reduce_sum(tf.square(grad_D_X_hat), reduction_indices=red_idx))` You'd...

WGAN-GP update formula wrong?

2

[Line 76](https://github.com/tdeboissiere/DeepLearningImplementations/blob/2163918fa2e2815aa0adf907ee56fd0897a75e9c/WGAN-GP/src/model/train_wgan_GP.py#L76) of train_wgan_GP: X_hat = X_real + epsilon * (X_fake - X_real) From the [paper](http://papers.nips.cc/paper/7159-improved-training-of-wasserstein-gans.pdf) this should be (Algorithm 1 Line 6): X_hat = epsilon * X_real + (...