ADOP Multi-gpu support

Hello, is there a way to run training on multiple gpus? have a machine with 4 gpus and was wondering if there was a config option(or plans) to enable multi-gpu support.

Thanks!

Apr 13 '22 05:04 parrot1166

Sorry this is currently not supported. You can however train multi instances on different GPUs, for example, to try out various settings.

May 05 '22 11:05 darglein

Sorry this is currently not supported. You can however train multi instances on different GPUs, for example, to try out various settings.

I have try to set the traning on different GPU like this: But, it still shows error and exits at training. and the GPU shows that it have memory can work. Is there something wrong？What else should I do?

Jul 11 '22 05:07 RobotBytedance

To train on a specific GPU you can use the CUDA_VISIBLE_DEVICES environment variable. Don't change the global "device" variable for that.

Jul 12 '22 13:07 darglein

To train on a specific GPU you can use the CUDA_VISIBLE_DEVICES environment variable. Don't change the global "device" variable for that.

Sorry for trouble you again. Where should I set the CUDA_VISIBLE_DEVICES？Could you please give me some resources or demo about that ?

Jul 12 '22 14:07 RobotBytedance