Swin-Transformer icon indicating copy to clipboard operation
Swin-Transformer copied to clipboard
verified publisher icon microsoft

DropPath: why scale_by_keep=True? eg: batch_id which not masked will multipy with 1./keep_prob, it may not make sense?

Open ignitemylife opened this issue 1 year ago • 0 comments

  • here is where it use drop path: https://github.com/microsoft/Swin-Transformer/blob/main/models/swin_transformer.py#L217

  • and here is timm implementation: https://github.com/huggingface/pytorch-image-models/blob/main/timm/layers/drop.py#L173

Should it set scale_by_keep=False explicitly?

ignitemylife avatar Aug 15 '24 08:08 ignitemylife