BEIT with Absolute Position Embedding

[hemangchawla]

Hello,

I would like to use BEIT model pre-trained with Absolute position embedding = True, and RPE ( incl. shared) = False.

However, I do not have the resources to train with an effective batch size of 2048.

Would it please be possible to provide appropriate pre-trained models on the repo?

Thanks and regards