fastembed

fastembed copied to clipboard

High memory usage when embedding large texts

12

Hi i am experiencing high memory usages which caused my pod to be killed because of exeeding its limits. after some experiments i found its related to text length that...

Barney241

Intro new sparse model based on attention weights. Consider it as an extension of bm25 for short documents.

generall

Model support request for BAAI/bge-m3

1

Leonschmitt

new: add clip exporter

1

joein

joein

Trying to download model from huggingface manually and want to use it from local path instead of download from HF

3

# Downloading model i.e, model.onnx and tokenizer.json, vocab.txt files from huggingface. Now I want to pass this local path and dont want fastembed to download from HF. trying below but...

visheshgitrepo

Add Support for BAAI/bge-m3

14

Canonical vector values: https://colab.research.google.com/drive/1OTZcLxWWAmJqV1ZDS57W5x6zJPRzku4n?usp=sharing Onnx model: https://huggingface.co/yashvardhan7/bge-m3-onnx/tree/main Contributed ONNX weights to the BAAI/bge-m3 HF model repo :https://huggingface.co/BAAI/bge-m3/tree/main/onnx

Ya-shh

Replace Data Source

2

Fixes #174

NirantK

Enforce sentence-transformers compatibility

3

@michaelfeil had mentioned this in https://github.com/qdrant/fastembed/pull/54 — but I kind of missed the bus on that. Let's do that now. Primarily, need to do two things: 1. Port this into...

NirantK

[MRL Support] dimensions with nomic-embed-text-v1.5

1

nomic-embed-text-v1.5 model supports variable dimensions, is there a way to set the model dimensions/size? embeddings_model = TextEmbedding("nomic-ai/nomic-embed-text-v1.5")

vontainment