pytorch
example time_sequence_prediction does not produce expected output
https://github.com/pytorch/examples/tree/master/time_sequence_prediction
After a few steps, the loss suddenly jumps up and stays high: loss: 0.00101216329969 loss: 0.195070432582
the pdf0 is OK, pdf1 is OK but from pdf2 the graph becomes an "L "
FYI: 4 years later and I'm having the same problem with time_sequence_prediction not being able to reproduce the expected results. My loss explodes after the 4th or 5th epoch. Changing the output layer to Linear, as is posed as the solution to this problem in #250, does not seem to fix this.
I'm guessing it's still a problem with learning rate being too high, but as the example doesn't work as it is currently written.
@rallen10 Thanks for raising this. I agree, this example's learning rate is (way) too high and the exploding loss resolves by setting it to something like 0.001.
I'll leave this open for a "good first issue"; along with fixing the learning rate, this example could also use a refactor ;)
Hi @subramen and @msaroufim , I've made the changes and created a pull request. Could one of you please review it?
