pinecone-io
Query vector dimension 76800 does not match the dimension of the index 768
I get the following error ApiException: (400) Reason: Bad Request HTTP response headers: HTTPHeaderDict({'content-type': 'application/json', 'date': 'Mon, 06 Mar 2023 17:34:00 GMT', 'x-envoy-upstream-service-time': '2', 'content-length': '110', 'server': 'envoy', 'connection': 'close'}) HTTP response body: {"code":3,"message":"Query vector dimension 76800 does not match the dimension of the index 768","details":[]}
For the code snippet:
import random
batch_size = 100 triplets = []
for i in tqdm(range(0, len(pairs), batch_size)): # embed queries and query pinecone in batches to minimize network latency i_end = min(i+batch_size, len(pairs)) queries = [pair[0] for pair in pairs[i:i_end]] print(len(queries)) pos_passages = [pair[1] for pair in pairs[i:i_end]] #print((pos_passages)) # create query embeddings query_embs = model.encode(queries, convert_to_tensor=True, show_progress_bar=False) # search for top_k most similar passages res = index.query(query_embs.tolist(), top_k=1) # iterate through queries and find negatives for query, pos_passage, query_res in zip(queries, pos_passages, res['results']): top_results = query_res['matches'] # shuffle results so they are in random order random.shuffle(top_results) for hit in top_results: neg_passage = pairs[int(hit['id'])][1] # check that we're not just returning the positive passage if neg_passage != pos_passage: # if not we can add this to our (Q, P+, P-) triplets triplets.append(query+'\t'+pos_passage+'\t'+neg_passage) break
the portion of your code that is creating the actual vectors is the issue where it is producing the 76800 value as dimensions when it should be generating vectors with dimensions of 768. It is multiplied by the batch_size
if the batch_size is set to 1. In this case, it throws an error in line "for query, pos_passage, query_res in zip(queries, pos_passages, res['results']):" saying the "ApiAttributeError: QueryResponse has no attribute 'results' at ['['received_data']']['results']"
if the batch_size is set to 1. In this case, it throws an error in line "for query, pos_passage, query_res in zip(queries, pos_passages, res['results']):" saying the "ApiAttributeError: QueryResponse has no attribute 'results' at ['['received_data']']['results']"
I'm having the same problem, have you solved it yet?
