QData
Evaluation arguments
Hi, I just finished training a Roberta model on IMDb like this
python train.py \
--train imdb \
--eval imdb \
--model-type roberta \
--model-save-path ./example \
--num-epochs 4 \
--num-clean-epochs 1 \
--num-adv-examples 0.2 \
--attack-epoch-interval 1 \
--attack a2t \
--learning-rate 5e-5 \
--num-warmup-steps 100 \
--grad-accumu-steps 1 \
--checkpoint-interval-epochs 1 \
--seed 42
and I tried to run the evaluation code
python evaluate.py \
--dataset imdb \
--model-type roberta \
--checkpoint-paths ./example \
--epoch 4 \
--save-log \
--accuracy \
--robustness \
--attacks a2t a2t_mlm textfooler bae pwws pso \
--interpretability
However, I got the following error
FileNotFoundError: [Errno 2] No such file or directory: './example/test_logs.json'
Did I miss something? In my example folder I can only see train_log.txt file with some checkpoint directories.
Hello, I'm encountering the same error. @Han8931, @qiyanjun, @jinyongyoo, @jakegrigsby could you please share your insights on how to resolve it?
@Yogender11 I'm sorry looks like there was a mistake with the file name. The file test_logs.json at https://github.com/QData/TextAttack-A2T/blob/91a20925858801cabeec2b89c598bb146e019b45/evaluate.py#L259 and https://github.com/QData/TextAttack-A2T/blob/91a20925858801cabeec2b89c598bb146e019b45/evaluate.py#L367 both refers to the json log file saved here https://github.com/QData/TextAttack-A2T/blob/91a20925858801cabeec2b89c598bb146e019b45/evaluate.py#L554
There must have been some last minute renaming that resulted in this issue. Could you check if accuracy_eval_logs.json file exists and if so, try to change the filenames at L259 and L367 and see if that fixes the problem?
