weightnorm icon indicating copy to clipboard operation
weightnorm copied to clipboard
verified publisher icon openai

Compatibility with Keras 2.2.3 and Tensorflow 1.11.0

Open krasserm opened this issue 7 years ago • 2 comments

This PR contains the required changes to get SGDWithWeightnorm and AdamWithWeightnorm running with Keras 2.2.3 and Tensorflow 1.11.0. I tested cifar10_cnn.py with both updated optimizers and training shows reasonable convergence. For testing the constraint branch in the optimizer code I temporarily added a kernel_constraint to the first Dense layer in cifar10_cnn.py (not part of the PR though) with reasonable results. I'm happy to include the test code in the PR if needed.

Update: It seems that this project is not active any more. I'll therefore continue maintaining the Keras 2.x version here. For more information about weight normalization implementation options for Keras see this blog post.

krasserm avatar Oct 03 '18 16:10 krasserm

@TimSalimans instead of changing the original implementation for Keras 1 I created a new keras_2 directory containing all changes needed for compatibility with Keras 2. The old keras directory was renamed to keras_1. Would appreciate your feedback on the PR.

krasserm avatar Oct 06 '18 06:10 krasserm

@TimSalimans you can see the actual changes from Keras 1 to Keras 2 in the second commit.

krasserm avatar Oct 06 '18 07:10 krasserm