Training On other Dataset Doesnot Seems Satisfactory

[ShoubhikBanerjee]

Hi, actually I was trying to finetune PreSumm on AmazonFoodReview Dataset for AbsBert. But after running nearly 3,00,000 training examples for 20,000 steps, The accuracy and the candidate summaries are not at all good.

Here are some of my accuracy per 1000 epocs : 1000 - 21 2000 - 19 3000 - 26 4000 - 23 5000 - 29 6000 - 22 7000 - 22 8000 - 20 9000 - 20 10000 - 22 11000 - 20 12000 - 21 13000 - 40 (best) 14000 - 16 15000 - 25 16000 - 18 17000 - 18 18000 - 17 19000 - 25 20000 - 22.

Could you please tell me whats the issue, and what is the actual use of "greedy" method in your code.?

What did actually went wrong with it?

[SebastianVeile]

What was your training command and how many gpus did you train on?

[ShoubhikBanerjee]

This was my command

!python PreSumm/src/train.py -task abs \ -mode train \ -bert_data_path 'Data/PreSummFinetune/AbstractiveBert/BertFormatted/formatted' \ -dec_dropout 0.2 \ -model_path 'drive/My Drive/models' \ -sep_optim true \ -lr_bert 0.002 \ -lr_dec 0.2 \ -save_checkpoint_steps 1000 \ -batch_size 14 \ -train_steps 20000 \ -report_every 5 \ -accum_count 5 \ -use_bert_emb true \ -use_interval true \ -warmup_steps_bert 200 \ -warmup_steps_dec 100 \ -max_pos 512 \ -visible_gpus 0 \ -log_file abs_bert_cnndm.log

And I was runnning it on GoogleColab with single GPU

[SebastianVeile]

The author describes here https://github.com/nlpyang/PreSumm/issues/44 that you need to adjust the accum_count when only training on one GPU

[gopalkalpande]

@ShoubhikBanerjee Canu plz let me know how u modeled Amazon Food Reviews dataset for training and which command u used for training the model?