Ali Saberi

Using (Absolute) Positional Embeddings with Hyena Operators

4

Hi @DanFu09 Hope you're well, I was reading the source code and the config files, and I realized that `use_positional_encodings` is `True` ([link](https://github.com/HazyResearch/m2/blob/b449ddbdbc9347ccf8e6332c1177affa1dec08f6/bert/yamls/embeddings/m2-bert-80M-32k-retrieval.yaml#L63)). So, the M2-BERT model is using an...