RuntimeError: /tmp/pip-req-build-cay04sa4/src/convolution_gpu.cu:66, assertion (at::cuda::check_device({in_feat, kernel})) failed. in_feat and kernel must be on the same device

[emilyemliyM]

Describe the bug A clear and concise description of what the bug is.

• Please complete all sections of this template if applicable. For installation, you must report the environment. Otherwise, your issue will be closed automatically.

---

To Reproduce Steps to reproduce the behavior. If the code is not attached and cannot be reproduced easily, the bug report will be closed without any comments.

• a minimally reproducible code.

---

Expected behavior A clear and concise description of what you expected to happen.

---

Desktop (please complete the following information):

• OS: [e.g. Ubuntu 20.04]
• Python version: [e.g. 3.8.5]
• Pytorch version: [e.g. 1.9.1]
• CUDA version: [e.g. 11.1]
• NVIDIA Driver version: [e.g. 450.11]
• Minkowski Engine version [e.g. 0.5.0]
• Output of the following command. (If you installed the latest MinkowskiEngine, paste the output of python -c "import MinkowskiEngine as ME; ME.print_diagnostics()". Otherwise, paste the output of the following command.)

wget -q https://raw.githubusercontent.com/NVIDIA/MinkowskiEngine/master/MinkowskiEngine/diagnostics.py ; python diagnostics.py

---

Additional context Add any other context about the problem here.

[TB5z035]

try setting the default device by torch.cuda.set_device(device) so that the kernel weights would be placed on the right device

#465