Oruli

Still nothing? Why is MultiGPU ignored in so many of these projects?

> > > Still nothing? Why is MultiGPU ignored in so many of these projects? > > > > > > Most likely, the impossibility of simple writing and debugging...

This has been open for a year, is there actually anyone on the project who can at least acknowledge it and add to a todo or something?

> Hi [@Oruli](https://github.com/Oruli) - we don't have any 5090s so we cannot test this but I do not see this on a machine with 2 A6000s. Could you perhaps share...

@loadams Still no reply even after my offer to debug and get this fixed? I'm not the only person with 2 x 5090s.

> [@Oruli](https://github.com/Oruli) - I re-read the thread. Are you still seeing this with the latest DeepSpeed version? Just so we can narrow our search to the current commits. > >...

@loadams Thanks for the help. So to be clear my setup is using Conda, I'm installing as follows to be able to run your script: ``` conda create -n deepspeed...

@loadams this is still an issue, I would really like to use the other 5090 i paid for, any chance we can get it resolved?

> [@Oruli](https://github.com/Oruli), I noticed in your OP that the failure occurs during a `send/recv` operation. Can you also try the p2p tests in the communication benchmark suite? https://github.com/deepspeedai/DeepSpeedExamples/tree/master/benchmarks/communication As I'm...

@loadams @tdrussell thank you for the help. Here is the output: ``` deepspeed --num_gpus=2 test.py --deepspeed_config ds_config.json [2025-07-17 09:15:53,939] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2025-07-17 09:15:56,497] [WARNING]...