umbertov

@liberbey hey you'll probably need a schedule with linear warmup for training any transformer, look [here](https://openreview.net/forum?id=B1x8anVFPr) for more info

Ok, thanks for the reply! Just for the sake of curiosity though, did you keep a record of the exact settings you trained with to reach the scores/obtain the checkpoints...

same problem here (only on windows), did you find a solution?

Hi @lucidrains I managed to get a stack trace of the error in the OP. I now know for sure that this happens for something wrong in the BYOL module,...

Hey, i can confirm that the culprit is the Kornia augmentations. I used `torchvision`'s ones and got rid of the problem. For anyone in my situation reading this in the...

A weird thing i noticed is that since we're supposed to maximize the alignment loss, and minimize the cross-model loss, is that the loss is quite prone to being negative,...

> it probably needs to be made aware of --target That's what i thought as well, I am unfamiliar with the codebase though, and I am not sure which callback...

Yes, I'm planning to have a look at it today, if I'm successful you'll find a pull request :smile:

**EDIT: Just found a workaround, see next message** Hello, I made the modification in [this](https://github.com/umbertov/gllvm/commit/cb1a5e6713a4686d20b0c0bc5b1b184ba636749d) commit. I added the following code: ```go "-target": {1, pr.compileLinkBinaryCallback}, ``` **Note**: the flag is...

Just found out that for some reason, `objcopy` (GNU toolchain) is the default, not `llvm-objcopy` from the LLVM toolchain. In order to get proper cross-compilation to work end-to-end, i need...