SwinTransformer
Strange output log
Hi authors, I have pretrianed your moby_swin_tiny model using 8 Tesla V100 GPU and reproduced your results in downstream task. I get 74.394% on linear evaluation and 43.1% on COCO object detection task, 39.3% on COCO segmentation task. But the loss and grad_norm is really weired during training. Can you show me your log? Here is my log. The loss drops to 7 and then rises to 16, then never drop again. During the pretraining task, the grad norm average value sometimes rises to infinite. log_rank0.txt
The uploaded txt log_rank0.txt is one of the eight gpus pretrain logs. And the uploaded txt log_rank7.txt is one of the eight gpus linear evaluation logs. log_rank7.txt
@launchauto @michuanhaohao me too, but I run it with precision O0. Did you run with the O0 precision? log_rank0.txt
怎么才能不适用apex混合精度呢？我使用swin transformer进行训练的时候，loss就会下降并且收敛。然而，我注意到swin transformer工程当中没有使用apex混合精度
Is it normal for the loss value to be around 16? Has anyone encountered this problem?
怎么才能不适合用apex混合精度呢？我用swin transformer进行训练的时候，loss就会下降并收敛。不过，我注意到swin transformer工程中没有使用apex混
请问您的问题解决了吗
Excuse me, have you solved the problem that loss drops to 8.9 and then rises in the opposite direction? Is it caused by apex mixed precision training?
