Jinhyuk Lee
Jinhyuk Lee
Could you check this issue? https://github.com/princeton-nlp/DensePhrases/issues/11
Hi, which version of DensePhrases are you using? And I think this might be related to the device (GPU) issue.
Oh I've missed that their batch size was 1. Thank you for your kind reply!
In this version of code, you cannot use sparse first search for `dev-v1.1` unless you have the document-level tfidf for SQuAD. See [here](https://github.com/uwnlp/denspi/issues/3). Thanks.
Hi, I think the problem is you forgot to give `--parallel`. The model is trained on DataParallel, so you have to give that option to load the model properly. Please...
Our model is mostly based on the neural network models whose behavior is difficult to predict. We hope to improve the consistency in the future. Thank you.
@mjeensung @donghyeonk do you have any idea for this?
The website is on again. Sorry for the inconvenience. We do not have a plan to use https for now. For a more secure connection, hosting a local server might...
``` $ python evaluate.py $SQUAD_DEV_PATH /tmp/piqa/pred.json {"exact_match": 53.207190160832546, "f1": 63.382281758599724} $ python piqa_evaluate.py $SQUAD_DEV_PATH /tmp/piqa/context_emb/ /tmp/piqa/question_emb/ {"exact_match": 53.39640491958373, "f1": 63.51748187339812} ``` Sometimes it goes up.
Have found that prediction json files are quite different, too. More than 600 preds out of 10570 preds are different between (original) pred.json and piqa version pred.json. One possible cause...