Giyeong Oh

Dadaptation optimizer

1

--use_dadaptation_optimizer args added. with this option, dadaptation is avalilable. while using dadaptation, it recommend 1-digit float numbers( 0.1

LearningRate-Free Learning Algorithm

33

Hi, how about [D-adaptation](https://github.com/facebookresearch/dadaptation)? This is a kind of algorithm that end-user doesn't need to set specific learning rate. In short, D-adaptation use boundedness to find proper learning rate. So,...

DeepSpeed Accelerator can NOT save optimizer state

3

Same as title. ~I'm in investigating.~ With following modification, scripts can save optimizer state **only for small dataset**, _IDK why it does not work big dataset._ Error messages will be...