NCE loss using positional params - swapped

[RadekSadowski]

I have used the code to run in Python notebook. Tensorflow 1.12.

I love this piece of implementation - straightforward step-by-step logic of CNN. It is so much more transparent and easy to adjust than today's Keras models.

One thing that caused an issue was loss function definition. I have got this error of type mismatch int32 vs float32. I found out that input and output params in tf.nn.nce_loss were swapped compared to 1.12 documentation: https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/nn/nce_loss

I wonder if your application worked because of change in tf api or maybe NCE loss is invariant in terms of input-output data? I could see people reported issue with exaggerate loss values - maybe it was related? Anyway, I have swapped them back and the code is now running.

So I suggest you adjust your code to use named parameters. Maybe it will save someone hustle in the future :)

self.loss = tf.reduce_mean(tf.nn.nce_loss(weights=softmax_w, 
                                                  biases=softmax_b, 
                                                  labels=self.y, 
                                                  inputs=h,
                                                  num_sampled=conf.num_sampled, 
                                                  num_classes=conf.vocab_size))

BR, Radek