selene icon indicating copy to clipboard operation
selene copied to clipboard

Early stopping and other parameters for learning rate scheduler

Open rfriedman22 opened this issue 3 years ago • 1 comments

Reference Issues/PRs

Fixes #183

What does this implement/fix? Explain your changes.

  1. Added support for other parameters for ReduceLROnPlateau. The TrainModel object has a new argument scheduler_kwargs that defaults to the pre-existing default parameters. User can specify other parameters as a dictionary, including through YAML config files. The kwargs get passed to the _init_train method when the scheduler is created. Support for other types of schedulers was not added.
  2. Added support for early stopping. The TrainModel object has a new optional argument stopping_criteria. If specified, it is a list that must correspond to the metric and patience for stopping criteria. When validate is called, it checks to see the last time the specified metric improves on the validation set. If the metric has not improved in the specified patience period, then the logger reports a debug message that early stopping has been reached and the for loop in train_and_validate is terminated.

What testing did you do to verify the changes in this PR?

  1. Created TrainModel objects from YAML config files that contain the new arguments. Verified the scheduler is assigned parameters specified in the config. Verified the early stopping criteria gets set as internal variables.
  2. I had previously implemented the early stopping code as a hack in my own scripts and verified that the criteria can be reached and stops training early. Code implemented here is virtually identical to what I had used in my scripts.

rfriedman22 avatar Jul 30 '22 18:07 rfriedman22

Thanks so much for this PR @rfriedman22 ! Is there an example you can add to config_examples or just a code snippet you could leave here for us to add to the website for this new change? LGTM (:

kathyxchen avatar Aug 15 '22 15:08 kathyxchen

Sorry for the delay @kathyxchen -- it's been a busy time lately. I've added an example config to this PR that demonstrates the new features :)

rfriedman22 avatar Sep 02 '22 18:09 rfriedman22