Sebastien Boisvert
Sebastien Boisvert
Can you submit a PR with your code snippet as the basis for a unit test, and possibly apply the proposed fix ?
Can you provide more details including: - small code example that reproduce the problem, - expected result - actual result ?
What is Expected behavior and Actual behavior ? I use xformers with a NVIDIA A40 on https://www.runpod.io/. Both the A40 and L40 have 48 GB VRAM.
Can you provide: - the version that you use - minimal-size code snippet - expected result - actual result
Hi @sagadre Here is my understanding. Before splitting your tensor into ``H`` heads, the shape of the tensor is ``[B, M, D]``, where ``B`` is batch size, ``M`` is sequence...
Hi again @sagadre It looks like you are right ! I looked at MultiHeadDispatch in xformers, which relies on RotaryEmbedding, and indeed it is used after the split into H...
Hi again @sagadre If you look at the unit test for rotary embedding, the input shape is (BATCH, HEADS, SEQ, EMB) and not (BATCH, SEQ, HEADS, EMB): https://github.com/facebookresearch/xformers/blob/748c159096d4f9fcfe3eaf22801e5aed4777210b/tests/test_rotary_embeddings.py#L61 So there...
Hi @arilato Is it the same mask everytime you call memory_efficient_attention ?