Hieu Pham issues

Repositories
Issues
Comments

Results 1 issues of


                                            Hieu Pham

Shouldn't the loss be normalized by the batch_size, not the number of words in a batch?

https://github.com/MultiPath/NA-NMT/blob/4054d606bf511e98aec82bc030034fa67dccc5f1/model.py#L976 [Luong et al., (2017)](https://github.com/tensorflow/nmt#loss) described this implementation in TensorFlow. In PyTorch, the [default behavior](http://pytorch.org/docs/0.3.1/_modules/torch/nn/functional.html#cross_entropy) is ``` size_average=True ignore_index=-100 reduce=True ``` which means that you are averaging the loss by...