tag device='cuda' not working

[arjunsbalaji]

Hi,

When I specify device=cuda I get an error saying cpu and cuda devices are in use?

I can see that in generate.py .to(device) is being called in inputs and model so not sure why this is happening.

[sarahalamdari]

Can you specify which script/function you are running? And I can help solve this

[colbyford]

@arjunsbalaji - When running from Python, you can run the following command to set the default Torch device:

import torch
torch.set_default_device('cuda:0')

This fixed the cpu/cuda error for me...

[yangkky]

@arjunsbalaji do you have an update or more details? Otherwise, I would like to close this.

[yangkky]

In general, I believe you have to set device='cuda:0', not just cuda.