openfl icon indicating copy to clipboard operation
openfl copied to clipboard

Envoy's Python environment degradation when do multiple experiments within the same Federation

Open alexey-gruzdev opened this issue 4 years ago • 0 comments

When we run multiple experiments (could be TF, then PyTorch, then old TF or PyTorch) within the same Federation, envoy's environment is started to degrade. Probably, the solution could be to create a separate environment for each experiment, or clean up the environment when experiment ended.

On Envoy when the Experiment is finished the gpu memory still allocated.

alexey-gruzdev avatar Sep 21 '21 15:09 alexey-gruzdev