oliver
oliver
Wonderful paper, a great experimental exposition of where/why the natural sequence models will cap out to support Weinsteins theoretical work. Could you provide the dataset used from Koenig? > Lastly,...
I ran ```bash export NEW_PROT_FASTA_FILE_PATH='/raid/app/oliver/tmp/disprot_2022_03.fasta' export NEW_PROT_RES_REPR_DIR_PATH='/raid/app/oliver/tmp/' ``` then the script (last bit of https://github.com/PeptoneInc/ADOPT/blob/main/scripts/adopt_inference.sh) ```bash TRAIN_STRATEGY="train_on_cleared_1325_test_on_117_residue_split" MODEL_TYPE="esm-1b" PRED_Z_FILE_PATH="predicted_z_scores.json" printf "Extracting residue level representation of %s \n" $NEW_PROT_FASTA_FILE_PATH python adopt/embedding.py...
Flash attention implemented to reduce runtime and memory usage using Pallas. Added on opt-in basis in the global config. For a 759 residue protein and model_5 this drops peak memory...
Matches the casting in multimer for monomers Runtime speedup of ~1.8x and reduces peak memory usage ~2x (for 759 residue protein, model_5) Here's a [colab](https://colab.research.google.com/drive/1YChuHrUs8D6Yf0jSb6Y4gGt23WGAY-8k?usp=sharing) link showing runtime improvement and...
Implements FlashAttention similarly to https://github.com/google-deepmind/alphafold/pull/931 For a 759 residue protein and model_5 this improves runtime 2.2x on an L4 (37.3 $\rightarrow$ 16.9 seconds [with minibatching of 256 for non-flash attention...
Matches behaviour in Triton where for load/store/swap any masked indexing does not occur. For load it sets any masked indexing to index to the first element in the array instead....
### Description For triton (if I have read this correctly) masked load/stores do not occur. So you can request to load/store to an index OOB for ref if that is...
(and adds stride support) Fixes https://github.com/google/jax/issues/21143 Implements a jittable masked gather/scatter where for load/store/swap any masked indexing does not occur. For load it sets any masked indexing to index to...