Z.X.
adaptive-inertia-adai
[ICML 2022, Oral] The PyTorch Implementation of Adaptive Inertia Methods. The algorithms are based on our paper: "Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum".
deep-learning-dynamics-paper-list
This is a list of peer-reviewed representative papers on deep learning dynamics (optimization dynamics of neural networks). The success of deep learning attributes to both network architecture and sto...
artificial-neural-variability-for-deep-learning
[Neural Computation, MIT Press] The PyTorch Implementation of Variable Optimizers/ Neural Variable Risk Minimization proposed in our Neural Computation paper: Artificial Neural Variability for Deep Le...
Positive-Negative-Momentum
[ICML 2021] The official PyTorch Implementations of Positive-Negative Momentum Optimizers.
stable-weight-decay-regularization
[NeurIPS 2023] The PyTorch Implementation of Scheduled (Stable) Weight Decay.