Adarsh Kumarappan
Adarsh Kumarappan
I believe the reason is twofold: 1. The "CC" environment variable may not be set on your system, so `distutils.sysconfig.get_config_vars()["CC"]` in the line mentioned above probably resulted in a `KeyError`....
I am using the same model checkpoint from HuggingFace for both datasets, so shouldn't the performance still be the same assuming that both datasets are on commit `29dcec074de168ac2bf835a77ef68bbe069194c5`?
So it seems that [v9](https://zenodo.org/records/10929138) was trained on commit `fe4454af900584467d21f4fd4fe951d29d9332a7`. Crucially, I tried generating this dataset on that commit and reproduced the same bug mentioned in the original post. Can...
>So you generated a new dataset from fe4454af900584467d21f4fd4fe951d29d9332a7 , evaluated the retriever model and got a Recall@10 of 36 or 70? 70. However, when I evaluated on the downloaded [v9](https://zenodo.org/records/10929138)...
I agree. However, I don't understand why I get ~70% when using a generated dataset from the same commit (`fe4454af900584467d21f4fd4fe951d29d9332a7`) used to train the model. Shouldn't I get ~36%? The...
Any thoughts @yangky11? Thank you so much for your help so far!
Thank you for your response, @yangky11. It doesn't give the same dataset, but the two datasets are close enough that the two R@10 values are essentially the same.
@yangky11 ?
Just wanted to follow up on this @RUrlus!
Sorry! I'll wait for your response.