Issue when applying AMP on LSTM

[rishikksh20]

Traceback (most recent call last): File "train.py", line 239, in args.warm_start, args.amp, args.name, hp) File "train.py", line 161, in train y_pred = model(x) File "/home/deepsyncv100-1/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/mnt/Karan/PPSpeech/model.py", line 66, in forward encoder_outputs = self.encoder(embedded_inputs, input_lengths) File "/home/deepsyncv100-1/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/mnt/Karan/PPSpeech/core/modules.py", line 183, in forward outputs, _ = self.lstm(x) File "/home/deepsyncv100-1/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/deepsyncv100-1/.local/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 580, in forward self.num_layers, self.dropout, self.training, self.bidirectional) RuntimeError: cuDNN error: CUDNN_STATUS_BAD_PARAM

[rishikksh20]

Fix is possible on next Pytorch release : https://github.com/pytorch/pytorch/issues/36428