Yulu Jia

use the hex representation of the tensor key in shm names. It's easier to tell the operation type, tensor id and partition number etc from the hex representation. Signed-off-by: yulu.jia

handle cases where some parameters are not used during the forward pass Signed-off-by: yulu.jia

add SyncBatchNorm

1

- add byteps.torch.SyncBatchNorm - add BYTEPS_SYNC_BN_GLOBAL to choose between global sync and node local sync. defaults to node local sync - add node local allgather Signed-off-by: yulu.jia