SPM_toolkit icon indicating copy to clipboard operation
SPM_toolkit copied to clipboard
verified publisher icon lanwuwei

about DecAtt

Open BruceLee66 opened this issue 6 years ago • 1 comments

When i use this model for wikiQA Task,i found that the batch list is difficult. image image Why should we resort the length?And The interval of batch_list is not 32.

BruceLee66 avatar Jul 19 '19 06:07 BruceLee66

DecAtt is very difficult to train, which I tried many ways to make it work, including gradient clipping, sorted length and etc. Previously people used length sorting to accelerate the model training and convergence speed, since the input doesn't vary a lot.

lanwuwei avatar Jul 20 '19 19:07 lanwuwei