syoya

> @kyquang97 Luong takes the last context vector and concatenates them with the last output vector as an input to RNN. The output from RNN will be passed to Attention...

> > > @kyquang97 Luong takes the last context vector and concatenates them with the last output vector as an input to RNN. The output from RNN will be passed...

@stgzr Thanks! I would have a try. And this seems strange to me. What's the difference between Pytorch 1.1.0 and 1.0.1 that could lead to this Nan loss?

Sorry for a late reply. I didn't read the notifications. I couldn't find the original paper where this kind of analysis is raised but I hope this one ([Why Regularized...

Interesting. I thought sparsity on representation means the same as sparsity on parameters. I'll try to figure it out. And sorry I'm quite busy these days.

> Added. According to issue #4 , the code refactor changed the randomness of the code. Please try different seeds if needed. [config](https://github.com/TencentYoutuResearch/Classification-SemiCLS/blob/main/configs/ccssl/fixmatchccssl_exp512_cifar10_wres_x2_b1x64_l4000_soft.py) I've tried several different seeds and finally...

Thanks for your reply! Here is what i use for training: **Training Command** `srun -p caif_dev --ntasks=1 --ntasks-per-node=1 --gres=gpu:1 --cpus-per-task=20 python train_semi.py --cfg configs/comatch/comatch_stl10_wres_r18_b1x64_l5.py --out workdirs/comatch_stl10_wres_r18_b1x64_l5 --seed 1 --gpu-id 0`...

hi @KaiWU5 here is my training log under torch1.6 environment and it seems that the results differ much compared with torch1.1x. I didn't complete the whole training process. Would you...

Yeah I nearly did nothing but simply change the torch version and GPU type (I'm now using 3090ti which doesn't support torch1.6_cuda10.1 so I rent a TITAN XP for experiment)....

I will keep this issue open to see if any conclusions could be reached. Also thanks for the cooperation on finding the cause!