Getting Nan as validation loss

[mobassir94]

was training a model with cyclic learning rate and after 7th epoch i get nan validation loss..isn't it "exploding gradient problem"? will gradient accumulation be able to solve this issue? i don't get such errors when i try adam or adamW. i am facing this problem whenever i try deepmemory or diffgrad optimizer,any help?

training statistics so far of the model i am trying :

Train Epoch: 0 LR: 0.0049400000 Loss: 1.878861 Dev loss: 2.1991 Train Epoch: 1 LR: 0.0086800000 Loss: 1.840539 Dev loss: 1.7849 Train Epoch: 2 LR: 0.0075800000 Loss: 1.847198 Dev loss: 1.9127 Train Epoch: 3 LR: 0.0038400000 Loss: 1.287447 Dev loss: 1.3331 Train Epoch: 4 LR: 0.0023000000 Loss: 1.416327 Dev loss: 1.2588 Train Epoch: 5 LR: 0.0060400000 Loss: 1.299999 Dev loss: 1.4838 Train Epoch: 6 LR: 0.0097800000 Loss: 1.540868 Dev loss: 1.5280 Train Epoch: 7 LR: 0.0064800000 Loss: 1.790969 Dev loss: 1.2738 Train Epoch: 8 LR: 0.0027400000 Loss: 1.092477 Dev loss: nan