rccl wrappers

Open avik-pal opened this issue 1 year ago • 1 comments

Similar to https://github.com/JuliaGPU/NCCL.jl/, it would be great to have wrappers for RCCL for training DL models on multiple AMDGPUs.

(I know MPI.jl has ROCm support but we don't ship JLLs that are rocm aware, so it might be good to ship RCCL_jll that automatically allows rocm aware communication without copying to CPU)

Jul 10 '24 19:07 avik-pal

+1. I was looking at Lux.jl and it's a shame that they have native support for NCCL but not for RCCL.

Dec 12 '24 17:12 ffrancesco94