Ryan XU
Ryan XU
> try fsdp or deepspeed I tried it and got > RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!...
> Are you sure the RGBA error is due to randomness? I ran it multiple times and got the error every single time
Are you seeing message like CUDA OOM --> CPU? I think when you run on a machine without enough vRAM, the model will be moved to CPU and weird things...