omkar-334
omkar-334
> each document is guaranteed to be used only once for question generation Do you mean document as in files or the node/embeddings?
> I can easily imagine that for longer documents you may need By document, they mean the current_nodes. I think the length of the document/file is irrelevant as it has...
> I'm not sure if explicit requirement to use each document only once is really what we need. I think you're right. But this is a recurring issue in the...
@sfc-gh-akashyap Check out this website - https://docs.ragas.io/en/stable/concepts/testset_generation.html It has a pipeline flowchart too.
@sfc-gh-akashyap The testset generator basically has this pipeline. 1. select a random node/embedding 2. generate a seed question using that 3. Get top_k embeddings similar to the random node selected...
Could the problem lie with `distributions` ? As far as I know, the sum of all distributions must add up to 1.
tag @joecummings
@pbontrager @krammnic @joecummings can you review this?
@mattdangerw @SamanehSaadat @divyashreepathihalli
@wuhaixu2016 These results contradict both the leaderboard rankings and the reported results in TimesNet paper. According to this, NS-Transformer beats Timesnet, but the paper shows otherwise...