Matteo Hessel

This looks very interesting, it would be an amazing contribution!

It's on my TODO list to think about ways for the rectification to be composable with different transforms. Hopefully I can get to this in the next couple of weeks,...

For instance instead of having a monolytic `scale_by_radam`, we could have a wrapper like this: ``` def rectify(inner: GradientTransformation b1: float = 0.9, b2: float = 0.999, threshold: float =...

The wrapped `GradientTransformation` would itself be a `GradientTransformation`. So you would first wrap the transform and then use the wrapped transform in a chain, e.g. you could do somethiing like...