Sentence Transformers Candidate Models

[NirantK]

Models

• [ ] https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2
• [ ] https://huggingface.co/sentence-transformers/multi-qa-MiniLM-L6-cos-v1
• [ ] https://huggingface.co/sentence-transformers/bert-base-nli-mean-tokens
• [ ] https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2
• [ ] https://huggingface.co/sentence-transformers/all-mpnet-base-v2
• [ ] https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1

Please add in comments if you would like a model from this list to be added!

[Tanmaypatil123]

@NirantK Hey, I think fastembed is dependent on the huggingface's tokenizer, right? Why does it not support any of the models mentioned in the issue description by default?

[NirantK]

We export the models to ONNX and quantize them to improve compute times, while preserving as much performance as we can — that means each model needs a bit of work and testing.

I am not sure how the tokenizer is relevant though. We don't use transformers or torch libs.

[Ya-shh]

Hey! I have used this https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2 in few chat-bots that i have built. And this really works well even with the large sized llms. You can also check in one where i used it: https://github.com/Ya-shh/Custom-Ai-ChatBot.git