deepcopy DistributedDataParallel loss actual model

Open meijieru opened this issue 6 years ago • 2 comments

model = nn.Linear(10, 2).cuda()
torch_wrapper = torch.data.parallel.DistributedDataParallel(model)
apex_wrapper = apex.parallel.DistributedDataParallel(model)

lhs = copy.deepcopy(torch_wrapper).module  # ok
rhs = copy.deepcopy(apex_wrapper).module  # failed, AttributeError: 'DistributedDataParallel' object has no attribute 'module'

Aug 23 '19 22:08 meijieru

@mcarilli any ideas on this ? I also face the same problem, I know torch DDP is recommended now, but any workarounds for this ?

Dec 17 '19 05:12 ASDen

Check out https://discuss.pytorch.org/t/torch-cuda-amp-vs-nvidia-apex/74994/2

Jan 07 '22 19:01 Peidong-Wang