loss is nan

Open lushizijizoude opened this issue 5 years ago • 1 comments

When training, the loss is nan，can you fix this problem？ Corpus can find in https://drive.google.com/drive/folders/1DuIszwD-2jDZwDuJ6Jjgf5k0a1jiXtcC?usp=sharing

Dec 22 '20 06:12 lushizijizoude

I have soloved the problem. It's due to the function log().

model.py

log_gate_score_1 = (gate_score+ 1e-45).log() # shape: (batch_size,) log_gate_score_2 = (1 - gate_score+ 1e-45).log() # shape: (batch_size,)

Dec 22 '20 06:12 lushizijizoude