Nils Reimers

I am also wondering what the response would be for a private model if the user is not authenticated? 404 or 401 Unauthorized? 404 would be more secure (so you...

This is an annoying thing about the elasticsearch client: They added a feature to the newest ES client which makes it work only with the newest ES server, to enforce...

I ran the script 3 times: ``` Run1 2021-04-20 16:10:20 - NDCG@1: 0.5400 2021-04-20 16:10:20 - NDCG@3: 0.6104 2021-04-20 16:10:20 - NDCG@5: 0.6297 2021-04-20 16:10:20 - NDCG@10: 0.6495 2021-04-20 16:10:20...

@NThakur20 Great, thanks for the quick fix. We should also either update the examples to the newest ES version, or freeze the Python ES client to e.g. version (in setup.py):...

Have a look at this code example: https://github.com/beir-cellar/beir/blob/c3334fd5b336dba03c5e3e605a82fcfb1bdf667d/examples/retrieval/evaluation/reranking/evaluate_bm25_ce_reranking.py#L63 It doesn't use the [EvaluateRetrieval.rerank](https://github.com/beir-cellar/beir/blob/c3334fd5b336dba03c5e3e605a82fcfb1bdf667d/beir/retrieval/evaluation.py#L25) method, instead it uses the [Rerank.rerank](https://github.com/beir-cellar/beir/blob/c3334fd5b336dba03c5e3e605a82fcfb1bdf667d/beir/reranking/rerank.py#L14) method.

As hostname I think you must use http://localhost (or http://localhost:9200), not just localhost

This was addressed in https://github.com/UKPLab/beir/issues/58 Not sure if the latest release already includes this. You can either update BEIR to use the latest version from the GIT. Or you add...

Hi @Muennighoff Thanks for reporting this. The spreadsheet was just an intermediate solution, not sure why Quora & CQA is missing. I currently work on an updated leaderboard that will...

@pommedeterresautee @Sharathmk99 I just pushed a clean and up-to-date example how to train BERT and other transformer model on MS Marco: https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/ms_marco/train_cross-encoder.py The result is a HuggingFace Transformer model that...

You can just set it to num_labels=2 and use it without further changes.