Condenser
Condenser copied to clipboard
luyug
EMNLP 2021 - Pre-training architectures for dense retrieval
Hi, The dev result of coCondenser on MSMARCO-Passage-Ranking-Submissions leaderboard is 0.443. Is it the results on Large size model ? Thank you @luyug 
Hi Luyu, Thank you for publishing this great work and the tools. Do you have/plans to make apis for Condenser, so that one can call your api in a case-wise...
Hi Luyu, @luyug Thanks for your interesting work! My question is how the title is used during your pre-training, especially in coCondenser pre-training? Is it just a span in the...
Hi, I'm newbie in Condenser. I use colab to run code and it can only run 24 hours max. I didn't find any resume_from_checkpoint in your code. How can i...
Hi Luyu, Could you guide where we can find the ICT pretrained model to replicate these results? Thanks
I pretrained a condeser-roberta-base on the same data and hyperparameters, but the results on downstream tasks were not high. Have you ever tried condenser pretraining on RoBERTa-base ? Thank you
The model checkpoints seem to be hard-coded as the BertForMaskedLM and are unable to load but to the CoCondensor class. Adding the following attributes in the initialization can surpass the...
Hello, In the paper it is stated that > ... given a random list of n documents [d1, d2, ..., dn], we extract randomly from each a pair of spans,...
Or could you describe your rerank set up like the learning rate, batch size, and how to choose negatives? Thanks a lot!
Hi, Thank you for your great work! I am willing to replicate your results on MS MARCO passage collection and I have a question regarding `Luyu/co-condenser-marco` model. Is this the...
