Stockfish icon indicating copy to clipboard operation
Stockfish copied to clipboard

Measure the spread in Elo performance by random seed

Open nodchip opened this issue 5 years ago • 1 comments

In https://github.com/nodchip/Stockfish/issues/152, we have measured Elo performance between nets generated by different commits and random seeds. Random seeds should affect the spread in Elo performance. We should measure the spread in Elo performance by setting random seed to several values.

nodchip avatar Sep 16 '20 06:09 nodchip

There are some combinations:

  1. Setting random seed to a constant value in training data generation, and setting random seed to a constant value.
  2. Setting random seed to a constant value in training data generation, and setting random seed to several values in training.
  3. Setting random seed to several values in training data generation, and setting random seed to a constant value.
  4. Setting random seed to several values in training data generation, and setting random seed to several values in training.

We should set the number of the threads to 1 to avoid the effects of multi-threading. We should also be better to measure the effects of single-threading vs multi-threading.

nodchip avatar Sep 16 '20 07:09 nodchip