stylable icon indicating copy to clipboard operation
stylable copied to clipboard
verified publisher icon wix

add SyncBatchNorm

Open pleasantrabbit opened this issue 4 years ago • 1 comments

  • add byteps.torch.SyncBatchNorm
  • add BYTEPS_SYNC_BN_GLOBAL to choose between global sync and node local sync. defaults to node local sync
  • add node local allgather

Signed-off-by: yulu.jia [email protected]

pleasantrabbit avatar Apr 16 '21 22:04 pleasantrabbit

For future reference: https://github.com/facebookresearch/fairscale/pull/662/files

eric-haibin-lin avatar May 09 '21 14:05 eric-haibin-lin