Rafael Josip Penić

@awaelchli Sadly, I cannot share the full code. However, I probably should have mentioned in my original post that I am using gradient checkpointing during training. After additional experimentation I...

@kylesargent Are you using gradient checkpointing? It seems that there was some sort of issue with gradient checkpointing in older Pytorch versions (

Hello, thank you for using RiNALMo! 😄 - Can you provide us with the script `/projects/p32327/RNAFOLD/RiNALMo-main/try.py` you are using to get RiNALMo's embeddings? Is it the same inference example code...

- What Pytorch version are you using? (You can check this with `python -c 'import torch;print(torch.__version__)'`) - Have you tried installing RiNALMo into "fresh" Conda environment? Something like this: ```bash...

Also, please check your gcc/g++ versions with `g++ --version` and `gcc --version`. According to [this issue](https://github.com/Dao-AILab/flash-attention/issues/224) you might want to update your gcc/g++ compiler if you have an outdated version...

I think it is very possible that outdated gcc version is creating problems for flash-attention package (hence the C code errors). Try updating `g++` version. Not sure how complicated that...

Hello, we are currently focused on some other aspects of the model and therefore downstream task inference script is not high on our priority list (at least right now). You...

Hello Marek! Hope you are doing well! Please check the `non_flash` branch in the repository. There we replaced FA mechanism with "ordinary" attention, which should be runnable on CPUs and...

(1) Ah, this is true. Currently, we use the rotary positional embedding implementation from the flash attention library. To resolve this issue, you need to either install flash-attention (which should...

Non-FA code on the main branch is legacy code that is not compatible with FA weights. As I said, please check the `non_flash` branch (`git checkout non_flash`) which contains a...