puck icon indicating copy to clipboard operation
puck copied to clipboard

Why normlize feature vector before searching in tinker?

Open xiaozxiong opened this issue 1 year ago • 2 comments

I found that there is a normlization operation before searching in tinker.

const float* feature = normalization(context.get(), request->feature);

What is the purpose of this operation? And when I used the default parameter whether_norm=true, I got a recall@100 of almost zero. After I changed it to whether_norm=false, the recall@100 was correct. Could you offer me some possible explanations?

Thank you!

xiaozxiong avatar May 14 '24 01:05 xiaozxiong

The default distance calculation method is cosine similarity. The returned distance value is obtained by applying the transformation '2 - 2 * cosine similarity' to the cosine similarity value between two vectors.

For other distance calculation methods, it is necessary to update the value of whether_norm to false.

nk2014yj avatar May 14 '24 02:05 nk2014yj

Thank you for your reply.

xiaozxiong avatar May 14 '24 02:05 xiaozxiong