serac icon indicating copy to clipboard operation
serac copied to clipboard

ES&DD in the run.log

Open BruthYU opened this issue 2 years ago • 0 comments

Thanks a lot for the opensource repo :star2: When the code performs validation (FC setting for example), run.log receives following information

[2023-06-15 10:05:09,444][trainer][INFO] - Step 0:
[2023-06-15 10:05:09,445][trainer][INFO] - sims/mean_edit_val:  0.55049; sims/pos_edit_val:  1.00000; sims/neg_edit_val:  0.00000; params/scale_edit_val:  0.00000; sims/mean_loc_val:  0.55074; sims/pos_loc_val:  1.00000; sims/neg_loc_val:  0.00000; params/scale_loc_val:  0.00000; cls/acc_val:  0.50000; cls/pos_acc_val:  1.00000; cls/neg_acc_val:  0.00000; loss/cntr_val:  0.69374; loss/cls_val:  1.39372; retain/edit_val:  0.28800; kl/edit_val:  0.35555; cntr/acc_val:  0.52019; cntr/log_prob_val: -0.69374; cntr/prob_val:  0.50277; time/edit_val:  0.01333; stat_dump/acc_val:  0.51137; stat_dump/log_prob_val: -0.69401; stat_dump/prob_val:  0.50179; stat_dump/nll_val:  0.69401; stat_dump/n_tokens_val:  6.88000; loss/total_supervised_val:  1.46310; memory/alloc_max_val:  11675536384.00000; memory/res_max_val:  14144409436.16000; loss/edit_val:  0.69401; loss/loc_val:  0.35555; edit/acc_val:  0.51137; edit/log_prob_val: -0.69401; edit/prob_val:  0.50179; loss/total_val:  0.42495; loss/total_edit_val:  0.42495; eval_time/elapsed:  32.82572; eval_time/average:  0.65651

I think the "Edit Success (ES)" is stat_dump/acc_val and the "Drawdown (DD)" is kl/edit_val Is that correct?

BruthYU avatar Jun 15 '23 03:06 BruthYU