Answers not related to facts

[rosieks]

When I try to search information I get the response that is completely not related to provided facts or it get information that it couldn't find the information. When I'm using Azure OpenAI with Own Data I get perfect answers. I wonder if there is a way to tune it somehow to have similar results. I'm using GPT 3.5 for text generation and Azure Cognitive Search. Although I noticed that while using Azure OpenAI with Own Data I use keyword search, not vector

[vvdb-architecture]

I noticed the same thing, using the file-based SimpleVectorDb .

The documents used as "ground facts" are the ones being "closest" to your query, where "closest" is defined using cosine similarity. I found that this kind of similarity often retrieves documents that have nothing to do with the subject at hand.

For example, queries like "How do I do X?" will retrieve documents where X (or something related to X) doesn't even occur.

I am investigating why this is. But this of course never happens with keyword search. Documents containing X would be retrieved by definition.

This is all very strange. The investigation continues.

You can find out which facts are being selected by placing a breakpoint at the call of SearchClient.GenerateAnswerAsyncin SearchClient.

[rosieks]

Actually in my case facts are correct but OpenAI ignores them completely.