main_loop_tf
main_loop_tf copied to clipboard
fvisin
A main loop based on dataset loaders
Uses latest `postfix` versatility of `tqdm>=4.22.0` (https://github.com/tqdm/tqdm/issues/382, https://github.com/tqdm/tqdm/pull/534)
The model checkpoints and tensorboard events are not saved with the same frequency. When the model is reloaded and training resumes this causes the main loop to write new events...
People tend to use `self.sess` in validation, which causes the hooks to be run. When the validation hook is among the hooks this causes an infinite loop. It's probably better...
Add a flag to optionally save runtime statistics every *n* batches to be visualized in tensorboard. See more information here: https://www.tensorflow.org/get_started/graph_viz#runtime_statistics
Image summaries are saved only in validation, we should add another summary for training as well.