Eder Santana comments

Results 81 comments of


                                            Eder Santana

Step by Step guide

Hi @mhbashari you can study the subjects in the order I showed them: 1) Calculus 2) Linear algebra 3) Statistics and Probability 4) Signal processing Lets assume you wanna take...

Loss sometimes goes to nan even with the gradient clipping

@carpedm20 in my NTM implementation (and in a couple of others I saw out there) `nan`s were usually caused by one of the following: - Initializing the memory to zero....

Loss sometimes goes to nan even with the gradient clipping

the `sharpening` value is is uses as `pow(input, sharpening)`. So it can't be negative. Use a nonlinearity like softplus to avoid getting negative values: `sharpening = tf.nn.softplus(sharpening)`.

Proposed README

Failure is unrelated to what I did. Maybe the Travis script needs an update.

Proposed README

btw, I got `out of memory` in a 5Gb GPU. Which gpus you guys used for that experiment?

Proposed README

@anuragranj would you consider adding the numbers you got here? https://github.com/EderSantana/blocks-examples/blob/patch-1/machine_translation/README.md Just edit it with Github itself.

Learning the initial weights of the heads leads to NaN

hi @tristandeleu, when learning the initial weights, are you making sure they are behind of a softmax? In other word, are you learning the initial logits instead? If so, there...

Learning the initial weights of the heads leads to NaN

I'm new to your codebase, could you point me out where you get the initial weights, I could try to check that out.

don't use hash_name but rather hash function directly

Hi @prusnak , Thanks for the PR. Comment: I think we could support both modes. Quickly passing a string makes the API easier to use and remember, while passing a...

don't use hash_name but rather hash function directly

Thanks! Let me leave this open then. If either somebody else picks this up or we get more such requests, we work on it later.