qdrant icon indicating copy to clipboard operation
qdrant copied to clipboard

Migrating from chromadb to qdrant for RAG creates inconsistencies

Open JINO-ROHIT opened this issue 1 year ago • 3 comments

I have been using llamaindex for RAG with chromadb, and the search metric was the L2 squared.

When I migrate my data to qdrant, the retrieved nodes are completely different despite following the same chunking strat. I noticed that Euclidean distance is the closest metric to L2 squared that takes a root. Even using this gives different results.

Is this entirely due to the search metric? Or is there something else going on?

Thanks

JINO-ROHIT avatar Sep 18 '24 13:09 JINO-ROHIT

Hey @JINO-ROHIT, if you compute the metric values manually, which version does it match?

generall avatar Sep 18 '24 19:09 generall

@generall it matches the chromadb version for L2 squared

JINO-ROHIT avatar Sep 19 '24 05:09 JINO-ROHIT

could you please share an example?

generall avatar Sep 19 '24 07:09 generall