kfac
kfac copied to clipboard
tensorflow
An implementation of KFAC for TensorFlow
Env: Python 3.10.13 tensorboard 2.15.1 tensorboard-data-server 0.7.2 tensorflow 2.15.0.post1 tensorflow-estimator 2.15.0 tensorflow-io-gcs-filesystem 0.34.0 Requirement already satisfied: h5py in /home/**/venv/lib/python3.10/site-packages (3.10.0) Requirement already satisfied: numpy>=1.17.3 in /home/**/venv/lib/python3.10/site-packages (from h5py) (1.26.2) Requirement...
I solved it by uninstalling tensorflow-probability==0.9.0 and installing 0.8.0. Observation: My TensorFlow version is 1.15. $ pip install kfac ``` Collecting kfac Downloading kfac-0.2.0-py2.py3-none-any.whl (178 kB) |████████████████████████████████| 178 kB 683...
According to the [docs](https://github.com/tensorflow/kfac/blob/master/kfac/python/keras/README.md), this optimizer is supposed to `converge much faster (>3.5x) and with fewer iterations (>14x) than SGD with Momentum`. In the K-fac vs Adam [example](https://github.com/tensorflow/kfac/blob/master/kfac/examples/keras/KFAC_vs_Adam_on_CIFAR10.ipynb), I encountered...
Hi, I ran the notebook given in the docs (KERAS KFAC example for CIFAR 10) , with the same network (Resnet-20) and parameters (tuned hyperparameters) and compared the first few...
I am trying to install and import K-FAC without success. I am using Anaconda, python 3.7.6 and have tried TensorFlow 2.1 and 1.15. Someone told me tf 1.5 might work,...
What is the best way to center the moving averages? If analytically our the activation kronecker factor is given by (a - \bar{a})^T(a - \bar{a}) where a are the instantaneous...
If we are reusing weights in a linear layer can we use the same approximation to compute the covariances, or are there some subtleties? for example if weights w are...
I was working with the RNN example, and tried to implement for some other problem using float64. I got an error for this, and it required float32 representation. Then it...
