Unable to find a valid cuDNN algorithm to run convolution

[OCEANOUXIN]

Hi! Thanks for your exellcent work.But when I run main_train.py,I got _Traceback (most recent call last):

_File ~/anaconda3/envs/nunet/lib/python3.8/site-packages/spyder_kernels/py3compat.py:356 in compat_exec exec(code, globals, locals)

File /home2/ouxin/3DUX-Net-main/main_train.py:252 global_step, dice_val_best, global_step_best = train(

File /home2/ouxin/3DUX-Net-main/main_train.py:198 in train loss.backward()

_File ~/anaconda3/envs/nunet/lib/python3.8/site-packages/torch/tensor.py:355 in backward return handle_torch_function(

File ~/anaconda3/envs/nunet/lib/python3.8/site-packages/torch/overrides.py:1394 in handle_torch_function result = torch_func_method(public_api, types, args, kwargs)

File ~/anaconda3/envs/nunet/lib/python3.8/site-packages/monai/data/meta_tensor.py:276 in torch_function ret = super().torch_function(func, types, args, kwargs)

_File ~/anaconda3/envs/nunet/lib/python3.8/site-packages/torch/tensor.py:1142 in torch_function ret = func(*args, **kwargs)

_File ~/anaconda3/envs/nunet/lib/python3.8/site-packages/torch/tensor.py:363 in backward torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)

File ~/anaconda3/envs/nunet/lib/python3.8/site-packages/torch/autograd/init.py:173 in backward Variable.execution_engine.run_backward( # Calls into the C++ engine to run the backward pass

RuntimeError: Unable to find a valid cuDNN algorithm to run convolution_

It seems that the loss.backward() gose wrong. Have you ever meet it? I don't know how to fix it. I have checked the CUDA 11.3 and cuDNN 8.2.0. If you know how to fix it, please let me know. Thanks.

[gg336699]

Hello,Have you solved this problem?

[BennettLandman]

I am closing the older bug reports as these were missed. We are now better tracking reports across the organization. Please re-open if this continues to be a blocker.