Lei Wang

> @tjruwase we ended up for the most part just using pure DDP in PyTorch. We did have moderate success using Fairscale which supported the variable batch sizes out of...

@tjruwase infinibatch may be a good choice for the dynamic batch in ddp. Notice, Dataset with the DistributedSampler may be better than infinibatch for validation set.

I think separating the language model from the evolocity can better decouple the code. In fact，i want to test our proposed model (https://github.com/ISYSLAB-HUST/ProtFlash). Now, evolocity is difficult to directly expand.

@Line290 Not yet，i guess the part parameters(Pretrained model) are not handled correctly.

hi @joelmeili, Can you show your example and error code? In theory, batchConverter does not take up a lot of memory.

hi， I found that you have a large number of fasta sequences, which may be caused by insufficient GPU memory. I recommend that you compute protien sequence embeddings in small...

@joelmeili Yes, I think your code is reasonable, but I suggest you can finetune the language model, which will bring huge benefits. Example: ``` model = your_model() flash_model = load_prot_flash_base()...

Hello, the pytorch lighting framework does not place the batch_token processed by the batchConverter function on the GPU, you need to implement it manually： ``` batch_token = batch_token.to(self.device) ``` If...

ESM-2 series models adopt RoPE positional encoding. In theory, ESM-2 can be used beyond the length limit requirement of 1024, but as the sequence length increases, the memory requirements are...