Channel API in Charm++ for direct GPU-GPU communication with UCX

Open minitu opened this issue 4 years ago • 1 comments

Implements the Channel API in Charm++ that allows the user to create a channel between a pair of chares and send/receive data using that channel (MPI-like). This aims at reducing the overheads from sending an extra metadata message & delaying the posting of the receive for incoming GPU data in the Zero Copy API based mechanism, by exchanging only data and not the flow of execution. This could be seen as trading off message-driven execution for better communication performance.

Stemmed from #3289.

Oct 22 '21 03:10 minitu

Tests successful on OLCF Summit: jsrun -n2 -a1 -c2 -g1 --smpiargs="-disable_gpu_hooks" ./bandwidth +ppn 1 +pemap L0 +commap L1 Need to add documentation.

Nov 15 '21 00:11 minitu