Is there any feature related to GPT-like models that can be applied to BERT-like models?

[zhangxin81]

Is there any fesature related to GPT-like models that can be applied to BERT-like models?

[byshiue]

They have some common optimization idea, like fusing the multi head attention kernel, quantizing the model to int8 or fp8.

[Ashwin-Ramesh2607]

hI @byshiue, a related question. Does BertAttentionPlugin also use FlashAttention2 that GptAttention uses?

[byshiue]

Yes.