I tried to deploy Mantis in my own server for some test. Do you have any suggestion about the tools which can deploy Mantis to run faster?

[BrenchCC]

I am trying to add Mantis to the supported model list in VLLM or Sglang

[jdf-prog]

For Mantis-Idefics2, it's literally the same as How Idefics2 is used. I found a existing issue in VLLM project that there are some people trying to implement Idefics2 in VLLM, I think probably you can try comment there to see what's the current progress.

For Mantis-CLIP and Mantis-SigLIP, they shall similar architectures with llava-1.5, which should be supported by VLLM now. You can try if that will work.

Inference acceleration is an important issue. I will also try to see if mantis can be integrated into VLLM in the near future.

[BrenchCC]

I had verified that Mantis-CLIP can be deployed by VLLM. But Mantis-SigLIP has a problem as follows: In the config of Mantis-SigLIP, image_size = 384 and patch_size = 14, but in VLLM , siglip.py requires a divisible relationship, and the 'LlavaForConditionalGeneration' in Mantis has difference compared with VLLM.