Arraymancer icon indicating copy to clipboard operation
Arraymancer copied to clipboard

softmax_backward missing

Open christopherzimmerman opened this issue 5 years ago • 1 comments

For Tensors stored on a CPU, there seems to be no implementation of softmax_backward as referenced here

I looked through the source when that file was added, and didn't find it in the code base then either. I am not that familiar with nim, so wanted to make sure I'm not missing an implementation somewhere before I worked on a PR.

christopherzimmerman avatar Oct 28 '20 14:10 christopherzimmerman

Seems like I missed adding this.

Usually you need fused Softmax and Negative log-likelihood which is implemented as softmax cross entropy here: https://github.com/mratsim/Arraymancer/blob/master/src/arraymancer/nn_primitives/nnp_softmax_cross_entropy.nim so feel free to implement softmax.

You can use this test as example on how to use numerical gradient for testing https://github.com/mratsim/Arraymancer/blob/master/tests/nn_primitives/test_nnp_loss.nim

Once nn_primitives are implemented and tested, I don't require testing for the higher level neural net for now.

mratsim avatar Oct 28 '20 16:10 mratsim