lbann icon indicating copy to clipboard operation
lbann copied to clipboard
verified publisher icon LLNL

Add distributed Scatter/Gather

Open szaman19 opened this issue 3 years ago • 0 comments

  • Enables distconv implementations of Scatter and Gather layers
  • Implements NVSHMEM based RMA kernels for scatter/gather on DiHydrogen tensors
  • Adds example applications in applications/graph/DistConvGNN/synthetic for benchmarking distributed Scatter, Gather, and GCN
  • Adds unit tests

To do:

  • [ ] Fix error in distconv identity layer causing mismatched mini-batch dimension

szaman19 avatar Jun 07 '22 16:06 szaman19