Leena Shekhar

Provided test set output is different from the output generated using the pre-trained models.

6

1. I used the pre-trained models (TF 1.2.1) to decode and calculated ROUGE scores for those. These outputs look different from the ones provided under the test_output folder. For example:...

Could you please make the trained models available?

Too many UNK in the output

2

I trained the network for 5000 iterations; see the loss below Building Graph Training started Model saved to disk at iteration #1000 val loss : 3.290424 Model saved to disk...

In cases where there is no explicit need for 2 vocabs (unlike NMT), should we have 2 separate vocabularies or just use one?

**else: if beam_search: return outputs[0], outputs[1], outputs[2:] # No gradient norm, loss, outputs.** I have a doubt here: why are you returning 3 values, when the comment says just 2...

Error while running the pretrained model on MNLI

9

Used the following command to run MNLI using the pretrained model: python run_classifier.py --do_train=False --do_eval=True --task_name=mnli_matched --data_dir=../MNLI/MNLI --output_dir=results --model_dir=model/xlnet_cased_L-24_H-1024_A-16 --uncased=False --spiece_model_file=model/xlnet_cased_L-24_H-1024_A-16/spiece.model --model_config_path=model/xlnet_cased_L-24_H-1024_A-16/xlnet_config.json --max_seq_length=128 --eval_batch_size=8 --num_hosts=1 --num_core_per_host=1 --eval_all_ckpt=False --is_regression=False It throws...