Christopher Beitel
Christopher Beitel
Still working on it! Hit some issues with my kube deployment when trying to re-deploy with accelerators. I'll push a version here in a minute to share for discussion.
So here's the [work in progress](https://github.com/cwbeitel/k8s/commit/dd7cd1935ecb2c79ab5682a5f14ddf4724da1f01), let me know if you guys have thoughts about [this](https://github.com/cwbeitel/k8s/tree/master/examples/agents) model for how model code is built into images and run. It doesn't have...
OK so here's a cleaner version of that PR https://github.com/tensorflow/k8s/pull/159. So next up is extending the example with some form of parallelization initially with the environments still running in the...
Thank you. Also made the example a bit simpler per @jlewi's suggestion, see https://github.com/cwbeitel/k8s/commit/790dfb1fd3ced4ec1b9cfd46221a666c4b1febdd. That sounds good about the tutorial happy to help user test those. You should be able...
Ok gs://agents-k8s-share/jobs/... should work going forward including gs://agents-k8s-share/jobs/tensorflow-20171117102413/20171117T182424-pybullet_ant Figures: Thanks happy to have help. Getting up to speed a bit on this class of algorithm and away this week. Maybe...
Agree this sounds like a standard instance of between-graph replication and the remaining task is to put the policy parameters on the parameter servers. Just looking for these and understanding...
Sounds good!
So just to update having some issues with variable initialization, current version [here](https://github.com/cwbeitel/k8s/tree/fc573e3024606cddaf10e20038259f6543698c44). Currently non-chief nodes hang, repeating: ```bash INFO:tensorflow:Waiting for model to be ready. Ready_for_local_init_op: Variables not initialized: ppo_temporary/episodes/Variable,...
Yeah me too. They don't appear to be on the param servers.