KataGo icon indicating copy to clipboard operation
KataGo copied to clipboard

Katago 40b v100 progress on cgos

Open hwj-111 opened this issue 4 years ago • 4 comments

I plot the progress trend of 40b network based on the data on cgos beyes elo Roughly improved at +0.2 elo / network image

hwj-111 avatar Nov 07 '21 00:11 hwj-111

Updated plot on 2022/02/10 image

hwj-111 avatar Feb 11 '22 05:02 hwj-111

I found now cgos BayesElo has accumulates enough b40v50 data points. So I plot both v100 and v50 together. v50 fitted line (slope ~ 0.15) seems lower by 180 elo than the v100 fitted line (slope ~ 0.19). This value seems a bit larger than the benefit of purely doubling the visits (+125 elo), indicating v100 is much preferred than v50 when using 40b networks (I guess, more visits needed before reach the 40b saturated limit)

image

hwj-111 avatar Aug 07 '22 06:08 hwj-111

Here is the plot of v100 (red) and v50 (green) of 60b network progress. The delta between them is about 170elo. The slope is much smaller than 40b ( only about ~0.14 per network# for v100, and 0.05 for v50!), indicating the progress is slower (when evaluate progress on v100). Maybe, v200 or v400 is much better ( large slope)?

image

hwj-111 avatar Aug 07 '22 16:08 hwj-111

I have ran some katago 60b nets on CGOS with visits/move = 200 (blue dots). Their progress is clearly better than v=100 (red) and 50 (green). Maybe, to see the actual progress, visits/move has to be higher, at least, 200 or so

image

hwj-111 avatar Oct 03 '22 04:10 hwj-111

update for katago 60b progress on CGOS, with vistts/move = 200, 100, and 50 image

hwj-111 avatar Dec 14 '22 03:12 hwj-111