Adding Support For Phi Models

[dawnik17]

Feature request

Adding support for phi models

Motivation

Currently phi models are not supported in bettertransformer. When trying to load a phi model, it raises NotImplemented error with a list of all the other implemented models here.

Your contribution

---

[Dev4011]

Yeah, it would be great if Microsoft Phi3 model can be supported using bettertransformer

[fxmarty]

Hi @dawnik17 @Dev4011 Transformers natively support scaled dot product attention operator from pytorch which was previously integrated through bettertransformer: https://github.com/huggingface/transformers/blob/e0dfd7bcaf7ff0723085f23244a755cc2ed92466/src/transformers/models/phi3/modeling_phi3.py#L614

What is not available in Transformers is the support of NestedTensor (or ragged tensor with a single dimension for both batch size/sequence length).

[miaoqiz]

I tried to use the "main_export()" to export a model, it shows the following error:

---

ValueError: Trying to export a custom model, but could not find as many custom ONNX configs as the number of submodels to export. Please specifiy the fn_get_submodels argument, that should return a dictionary of submodules with as many items as the p

---

What can be done to resolve this issue?

[Dev4011]

@fxmarty so what is the tentative timeline you are considering for supporting Phi3 models via betterTransformers?

[github-actions[bot]]

This issue has been marked as stale because it has been open for 30 days with no activity. This thread will be automatically closed in 5 days if no further activity occurs.