Is it possible to add support for Kosmos-2 Model

[VishalBCA-ops]

Hello Coremltools Team,

I was just checking if It is possible for me to convert "https://huggingface.co/microsoft/kosmos-2-patch14-224" model to support coreml so that I can use it on my mac?

its an Image to Text (Image Captioning Model)

I have tried it now but I could not find any helpful things, Is there any way I or we could add support for this?

Thanks!!!!

❓Question

• If this is a question about the Core ML Frame work or Xcode, please ask your question in the Apple Developer Forum: https://developer.apple.com/forums/

[YifanShenSZ]

Hi @VishalBCA-ops, many thanks for bringing the awesome kosmos model to our attention! I think this model is in PyTorch format, so probably you can start with trying our PyTorch converter

[Suraj209211]

@VishalBCA-ops, Hello, I saw the Kosmos model basically it's a mllm model that predict what happen next once the request is generated by the user. Basically model trained on Transformers. text to image based generation.

So I think it's possible to add the support of Kosmos model in CoreML but need to convert the entire thing using onnx-coreml package.

Lemme try using the hugging face code and let you know soon abt this