cuad icon indicating copy to clipboard operation
cuad copied to clipboard

Why is test dataset (test.json) labeled?

Open ShuJackson opened this issue 4 years ago • 1 comments

The "--predict_file ./data/test.json" file is labeled with questions and answers, and it's passed directly into predictions = compute_predictions_logits() for predictions in train.py.

If I want to use your model to do predictions on my own dataset, do I also need to label it in the same json format? Doesn't that defeat the purpose? Let me know if I am misunderstanding, but shouldn't the model predict on unlabeled, raw text file?

Thanks!

ShuJackson avatar Jun 10 '21 17:06 ShuJackson

The "--predict_file ./data/test.json" file is labeled with questions and answers, and it's passed directly into predictions = compute_predictions_logits() for predictions in train.py.

If I want to use your model to do predictions on my own dataset, do I also need to label it in the same json format? Doesn't that defeat the purpose? Let me know if I am misunderstanding, but shouldn't the model predict on unlabeled, raw text file?

Thanks!

@ShuJackson I'm also facing this issue. Were you able to figure it out?

berikohen avatar Jan 31 '22 23:01 berikohen