Kian Kim

Code for search & train WITHOUT weight sharing

3

The authors report the results for the random search baseline without any weight sharing but do not provide the code to reproduce the baseline's result. This would be helpful in...

First of all, thank you for your hard work in providing the community with such a good framework! I've been digging through many binary network papers & repos because I...

No L2 Normalization for the output of the encoder network?

9

In the paper it is described that L2 normalization is performed on the output of the encoder as well as the output of the projection head. However, in the code...

Huge accuracy drop from AR with 1 refinement to NAR with 2 refinements

1

I was testing the model using custom STR data and found that there is a huge accuracy gap (up to 20+% word accuracy) going from AR with 1 refinement to...

If you look [here](https://github.com/microsoft/Swin-Transformer/blob/main/models/swin_transformer_v2.py#L128), only q, v biases are initialized as parameters and k bias is created during the forward [pass](https://github.com/microsoft/Swin-Transformer/blob/main/models/swin_transformer_v2.py#L149) with requires_grad=False. It would be much appreciated if some...

PyTorch Implementation: Forward Pass

2

First of all, thank you for sharing the PyTorch implementation, it's wonderful. I've been going over the code and found this line: binary_weights = binary_weights_no_grad.detach() - cliped_weights.detach() + cliped_weights in...