Evaluation every 10k steps

Open adrienohana opened this issue 3 years ago • 2 comments

Hello,

I can't figure out how to use the evaluation while training, not sure what the data format is and how to plot the ndcg@k. I've figured out how to do it after training though, by loading the saved models in a loop and predicting on my evaluation data.

The question I have is, does seeing the evaluation only every 10k steps make sense ? How to be sure there aren't some big variations in between ? My training doesn't stop improving after 100k steps and is not as smooth as in your paper. Screenshot 2023-01-31 at 14 31 01

Any hints would be greatly appreciated.

Jan 31 '23 10:01 adrienohana

@nreimers @kwang2049 Would love your help with this. Need this for my implementation as well!

Aug 18 '23 12:08 shreyasavant

@adrienohana Did you find any way to evaluate during training ?

May 07 '24 14:05 KuijpersNick0